-- Logs begin at Mon 2022-09-05 11:34:02 UTC, end at Mon 2022-09-05 12:47:52 UTC. -- Sep 05 12:04:18.705035 dv-cinder-855189-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 05 12:04:18.719291 dv-cinder-855189-1-iscsi devstack@keystone.service[81528]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 05 12:04:18.723763 dv-cinder-855189-1-iscsi devstack@keystone.service[81528]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Sep 5 12:04:18 2022] *** Sep 05 12:04:18.723984 dv-cinder-855189-1-iscsi devstack@keystone.service[81528]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 05 12:04:18.724140 dv-cinder-855189-1-iscsi devstack@keystone.service[81528]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 05 12:04:18.724305 dv-cinder-855189-1-iscsi devstack@keystone.service[81528]: nodename: dv-cinder-855189-1-iscsi Sep 05 12:04:18.724462 dv-cinder-855189-1-iscsi devstack@keystone.service[81528]: machine: x86_64 Sep 05 12:04:18.724639 dv-cinder-855189-1-iscsi devstack@keystone.service[81528]: clock source: unix Sep 05 12:04:18.724798 dv-cinder-855189-1-iscsi devstack@keystone.service[81528]: pcre jit disabled Sep 05 12:04:18.724956 dv-cinder-855189-1-iscsi devstack@keystone.service[81528]: detected number of CPU cores: 4 Sep 05 12:04:18.725107 dv-cinder-855189-1-iscsi devstack@keystone.service[81528]: current working directory: / Sep 05 12:04:18.725269 dv-cinder-855189-1-iscsi devstack@keystone.service[81528]: detected binary path: /usr/bin/uwsgi-core Sep 05 12:04:18.725269 dv-cinder-855189-1-iscsi devstack@keystone.service[81528]: your processes number limit is 47680 Sep 05 12:04:18.725269 dv-cinder-855189-1-iscsi devstack@keystone.service[81528]: your memory page size is 4096 bytes Sep 05 12:04:18.725269 dv-cinder-855189-1-iscsi devstack@keystone.service[81528]: detected max file descriptor number: 2048 Sep 05 12:04:18.725269 dv-cinder-855189-1-iscsi devstack@keystone.service[81528]: lock engine: pthread robust mutexes Sep 05 12:04:18.725585 dv-cinder-855189-1-iscsi devstack@keystone.service[81528]: thunder lock: enabled Sep 05 12:04:18.725805 dv-cinder-855189-1-iscsi devstack@keystone.service[81528]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 05 12:04:18.725924 dv-cinder-855189-1-iscsi devstack@keystone.service[81528]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 05 12:04:18.766819 dv-cinder-855189-1-iscsi devstack@keystone.service[81528]: Python main interpreter initialized at 0x55c1e83299c0 Sep 05 12:04:18.766819 dv-cinder-855189-1-iscsi devstack@keystone.service[81528]: python threads support enabled Sep 05 12:04:18.766819 dv-cinder-855189-1-iscsi devstack@keystone.service[81528]: your server socket listen backlog is limited to 100 connections Sep 05 12:04:18.766819 dv-cinder-855189-1-iscsi devstack@keystone.service[81528]: your mercy for graceful operations on workers is 90 seconds Sep 05 12:04:18.767384 dv-cinder-855189-1-iscsi devstack@keystone.service[81528]: mapped 403077 bytes (393 KB) for 2 cores Sep 05 12:04:18.767384 dv-cinder-855189-1-iscsi devstack@keystone.service[81528]: *** Operational MODE: preforking *** Sep 05 12:04:18.767642 dv-cinder-855189-1-iscsi devstack@keystone.service[81528]: *** uWSGI is running in multiple interpreter mode *** Sep 05 12:04:18.767642 dv-cinder-855189-1-iscsi devstack@keystone.service[81528]: spawned uWSGI master process (pid: 81528) Sep 05 12:04:18.767838 dv-cinder-855189-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 05 12:04:18.771563 dv-cinder-855189-1-iscsi devstack@keystone.service[81528]: spawned uWSGI worker 1 (pid: 81541, cores: 1) Sep 05 12:04:18.771563 dv-cinder-855189-1-iscsi devstack@keystone.service[81528]: spawned uWSGI worker 2 (pid: 81542, cores: 1) Sep 05 12:04:18.771563 dv-cinder-855189-1-iscsi devstack@keystone.service[81528]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 05 12:04:23.189494 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 05 12:04:23.189494 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: from cryptography.utils import int_from_bytes Sep 05 12:04:23.190771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 05 12:04:23.190771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: from cryptography.utils import int_from_bytes Sep 05 12:04:23.193427 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 05 12:04:23.193427 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: from cryptography.utils import int_from_bytes Sep 05 12:04:23.194208 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 05 12:04:23.194208 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: from cryptography.utils import int_from_bytes Sep 05 12:04:23.243735 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 05 12:04:23.243988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 05 12:04:23.244109 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 05 12:04:23.244261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] command line args: [] {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 05 12:04:23.244388 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 05 12:04:23.244540 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 05 12:04:23.244540 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ================================================================================ {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 05 12:04:23.244749 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] command line args: [] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 05 12:04:23.244875 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] admin_token = **** {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.244987 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] config_dir = [] {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.245094 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 05 12:04:23.245215 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.245325 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] config_source = [] {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.245433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ================================================================================ {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 05 12:04:23.245557 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] debug = True {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.245678 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.246554 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] admin_token = **** {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.246776 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.247020 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] config_dir = [] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.247243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.247243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.247755 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.248114 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.248453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] config_source = [] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.248791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] list_limit = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.248791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] log_config_append = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.249406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] debug = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.249745 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.249745 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] log_dir = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.250360 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.250759 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] log_file = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.250759 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.251953 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.252292 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.252635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.252873 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.252873 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.253173 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.253356 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.253356 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.253356 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.253775 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.253908 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.254092 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] list_limit = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.254222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.254222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.254472 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] log_config_append = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.254607 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.254745 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.254870 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.254870 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.255110 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] log_dir = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.255308 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.255546 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] log_file = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.255698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.255828 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.255968 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.255968 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] publish_errors = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.256226 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.256355 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.256355 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.256588 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.256715 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.256715 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.256971 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.257113 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.257248 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.257375 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] standard_threads = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.257375 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.257625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.257759 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.257759 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.257988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.258112 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] use_journal = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.258112 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] use_json = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.258112 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] use_stderr = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.258469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.258598 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] use_syslog = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.258598 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.258825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.258969 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.258969 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.259238 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.259375 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.259539 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.259681 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.259681 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.259981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.260122 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.260122 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.260468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.260599 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.260599 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.260828 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.260955 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.260955 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.261260 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.261402 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.261402 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.261402 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.261402 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.261861 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.262000 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.262129 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] publish_errors = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.262267 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.262267 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.262509 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.262645 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.262645 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.262884 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.263012 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.263140 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.263280 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.263280 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.263566 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.263697 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.263825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.263947 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.263947 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.264202 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] standard_threads = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.264338 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.264338 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.264579 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.264716 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.264716 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.264972 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.265100 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.265236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.265361 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.265361 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.265620 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] use_journal = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.265756 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.265892 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] use_json = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.266029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.266029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.266267 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] use_stderr = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.266546 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.267169 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] use_syslog = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.267298 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.267298 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.267592 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:04:23.267738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.267738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.267990 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.268118 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.268118 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.268358 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.268495 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.268495 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.268745 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.268881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.268881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.269128 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.269253 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.269253 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.269507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.269634 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.269634 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.269864 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.269998 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.270140 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.270269 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.270269 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.270511 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.270647 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.270647 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.270893 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.271018 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.271018 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.271249 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.271377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.271377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.271703 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.271833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.271833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.272093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.272231 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.272231 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.272594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.272726 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.272726 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.272957 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.273096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.273096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.273345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.273472 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.273472 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.273706 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.273844 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.273983 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.274111 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.274111 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.274344 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.274474 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.274474 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.274717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.274845 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.274845 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.275079 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.275207 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.275207 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.275457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.275603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.275603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.275811 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.275922 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.275922 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.276143 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.276254 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.276254 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.276452 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.276564 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.276564 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.276842 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.276955 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.276955 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.277165 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.277279 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.277279 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.277492 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.277604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.277728 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.277849 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.277849 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.278056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.278181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.278181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.278393 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.278522 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.278522 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.278723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.278836 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.278836 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.279050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.279160 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.279160 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.279711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.279825 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.279825 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.280048 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.280167 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.280277 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.280387 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.280387 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.280602 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.280713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] auth.external = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.280713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.280916 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.281029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.281141 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.281264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.281376 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] auth.password = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.281560 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] auth.token = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.281777 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.281982 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.281982 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.282396 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.282704 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.282838 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.282970 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.282970 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.283272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.283416 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.283416 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.283740 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.283873 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] credential.caching = True {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.283873 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.284110 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.284267 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.284267 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.284501 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.284630 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.284759 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.284889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.284889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.285135 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.285265 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.285265 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.285500 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.285612 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.285737 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.285872 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.286045 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.286220 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.286356 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.286542 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.286791 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.286938 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.287079 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.287209 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.287344 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] federation.caching = True {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.287473 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.287625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.287773 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.287960 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.288098 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.288229 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.288417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.288552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.288702 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.288829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.288959 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.289091 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.289214 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.289345 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.289580 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.289717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.289847 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.290012 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.290159 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] identity.caching = True {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.290300 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.290474 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.290610 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.290741 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.290903 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.291088 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.291229 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.291340 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.291453 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.291719 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.291851 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.291982 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.292126 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.292255 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.292404 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.292561 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.292692 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.292930 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.293061 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.293192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.293323 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.293466 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.293601 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.293734 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.293863 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.293991 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.294222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.294369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.294502 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.294681 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.294816 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.294947 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.295135 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.295285 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.295415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.295621 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.295760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.295891 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.296065 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.296215 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.296348 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.296489 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.296621 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.296763 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.296921 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.297072 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.297239 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.297371 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.297512 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.297760 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.297910 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.298055 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.298218 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.298353 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.298485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.298671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.298808 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.298953 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.299083 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.299221 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.299352 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.299612 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.299757 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.299909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.300039 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.300226 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.300394 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.300531 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.300662 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.300827 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.300966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.301158 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.301293 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.301424 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.301626 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.301775 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.301905 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.302084 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.user = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.302221 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.302352 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.302519 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.302671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.302816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.302974 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.303110 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.303239 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.303485 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.303708 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.304218 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.304415 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.304600 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.304600 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.304969 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.305153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.305153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.305509 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.305694 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.305694 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.306050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.306238 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.306423 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.306607 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.306607 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.306951 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.307147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.307147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.307510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.307725 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.307725 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.308073 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.308300 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.308300 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.308643 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.308824 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.308824 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.309168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.309372 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.309556 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.309748 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.309748 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.310093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.310276 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.310276 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.310632 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.310813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.310813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.311154 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.311336 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.311336 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.311768 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.311949 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.311949 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.312291 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.312476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.312476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] resource.caching = True {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.312834 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.313016 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.313016 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.313852 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] auth.external = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.314053 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.314053 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.314406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.314598 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.314598 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.314941 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.315125 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.315125 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.315480 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.315682 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.315682 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] role.caching = True {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.316019 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] auth.password = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.316202 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] role.driver = sql {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.316402 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] auth.token = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.316586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.316586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.316931 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.317114 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.317114 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.317470 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.317650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.317650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.318000 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.318176 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.318176 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.318537 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.318721 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.318721 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.319063 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.319246 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.319246 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.319808 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.319998 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.320182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.320363 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.320545 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] credential.caching = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.320727 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.320727 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.321082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.321260 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.321260 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.321607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.321802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.321802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.322146 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.322327 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.322327 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.322666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.322861 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.322861 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.323250 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.323433 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.323433 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.323804 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.324004 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.324004 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.324349 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.324529 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.324529 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.324870 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.325068 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.325068 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.325410 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.325592 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.325592 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] token.caching = True {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.325942 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.326139 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.326139 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.326481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.326661 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.326661 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.327003 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.327203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.327203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.327566 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.327750 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.327750 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.328094 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.328301 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.328301 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.328644 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.328827 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.328827 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.329806 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] federation.caching = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.330416 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.330607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.330790 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.330790 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.331148 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.331344 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.331344 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.331832 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.332018 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] database.connection = **** {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.332018 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.332358 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.332542 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.332542 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.332906 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.333091 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.333091 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.333431 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.333616 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.333616 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.333971 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.334156 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.334156 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.334496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.334678 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.334678 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.335031 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.335213 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.335213 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.335581 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.335767 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.335767 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.336129 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.336314 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.336314 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.336660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity.caching = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.336840 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.336840 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 05 12:04:23.337292 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.337292 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.337292 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.337292 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.337292 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.337292 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.337292 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.337292 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.337292 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.338842 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.338842 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.338842 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.338842 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.338842 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.338842 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.338842 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.338842 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.338842 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.340357 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.340357 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.340357 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.340357 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.341027 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.341027 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.341378 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.341742 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.342112 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.342463 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.342829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.343196 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.343585 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.343966 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.344331 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.344728 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.345085 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.345433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.345795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.346148 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.346507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.346860 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.347213 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.347761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.348149 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.348506 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.348857 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.349225 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.349594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.349951 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.350313 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.350670 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.351018 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.351377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.351781 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.352134 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.352507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.352873 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.353241 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.353593 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.353940 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.354298 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.354650 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.355007 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.355354 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.355771 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.356143 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.356515 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.356879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.357246 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.357592 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.357954 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.358306 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.358668 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.359021 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.359373 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.359766 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.360134 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.360494 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.360717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.360906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.361096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.361285 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.361468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.361785 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.362146 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.362499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.362851 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.363214 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.363613 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.363964 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.364327 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.364685 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] resource.caching = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.365053 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.365419 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.365772 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.366144 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.366516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.366867 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.367231 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.367620 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.368058 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.368384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] role.caching = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.368747 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] role.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.369085 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.369471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.369610 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.369885 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.370272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.370648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.371029 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.374593 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.374593 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.374593 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.374593 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.374593 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.374593 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.374593 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.374593 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.374593 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.377098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.377098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.377098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.377098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.377098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.377098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.377098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.377098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.377098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.378139 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.378139 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.378139 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.378139 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.378616 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.378909 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.379215 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.379512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.379847 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.380148 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] token.caching = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.380449 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.380762 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.381061 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.381381 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.381691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.381988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.382294 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.382600 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.382914 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.383209 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.383509 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.383843 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.384150 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.384490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.384778 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.385086 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.385416 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.385739 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.connection = **** {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.386044 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.386352 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.386668 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.386955 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.387234 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.387513 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.387827 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.388107 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.388395 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.388673 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.388961 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.389244 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.389538 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.389814 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.390102 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.390373 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.390659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.390977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:04:23.391192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 05 12:04:23.399432 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.399817 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.419901 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.420243 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.420461 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.429799 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.449392 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.449718 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.457286 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.489327 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.512777 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.513162 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.525996 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.526347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.588114 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.588486 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.588696 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.588909 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.588998 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.589172 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.617566 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.617826 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.618032 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.618174 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.618399 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.618610 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.618829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.673402 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.673671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.673874 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.674050 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.674234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.674447 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.688156 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.689095 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.689313 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.689460 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.689591 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.689771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.689982 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.702444 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.715876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.716124 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.716318 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.716542 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.716759 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.716978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.717178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.722292 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.725246 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.725471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.725693 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.725916 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.726133 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.726347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.726561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:04:23.734195 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.734424 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 05 12:04:23.734728 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.734923 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 05 12:04:23.735740 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.736021 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.736279 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.736536 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.737159 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.737365 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 05 12:04:23.737712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.737907 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 05 12:04:23.738182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.738441 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.738701 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.738957 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.746182 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.746469 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.746737 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.747021 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.748477 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.748765 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.749038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.749318 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.754626 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.757024 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.757563 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.757846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.758127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.758405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.758681 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.758938 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.759203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.759473 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.759769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.759911 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.760091 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.760265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.760457 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.760630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.760799 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.761021 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.761282 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.761559 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.761836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.762109 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.762382 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.762657 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.783930 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.785426 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.785702 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.786196 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.787723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.788011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.790433 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.790727 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.791010 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.792736 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.793026 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.793314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.797444 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.798325 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.800335 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.800636 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.804800 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.805097 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.805388 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.805677 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.805955 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.806231 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.806397 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.806503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.806696 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.806837 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.806973 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.807247 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.807554 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.807820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.808085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.808360 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.822273 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.822549 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.823968 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.824270 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.826393 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.828204 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.829689 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.829994 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.831766 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.832115 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.834296 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.836402 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.837433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.839562 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.840673 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.840977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.841261 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.841543 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.841819 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.842098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.842879 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.843195 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.843482 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.843796 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.844075 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.844369 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.860056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.860898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.861181 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.861441 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.861592 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.861786 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.862123 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.862293 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.868378 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.869948 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.870112 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.871465 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.871699 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.871844 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.873503 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.873780 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.874406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.874710 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.874997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.875280 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.875704 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.876941 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.877228 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.877519 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.877798 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.878198 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.886915 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.887207 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.887492 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.887805 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.888195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.888477 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.888758 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.889910 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.890209 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.890492 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.890772 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.891169 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.891486 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.891794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.904291 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.908035 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.908249 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.910929 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.911099 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.912402 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.913884 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.914075 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.914220 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.914562 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.915344 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.916912 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.917196 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.917613 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.921197 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.923415 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.925045 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.926494 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.926697 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.926931 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.927222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.927503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.929637 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.930054 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.930340 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.930618 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.935427 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.935755 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.936038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.938613 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.938926 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.939222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.943222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.943548 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.943827 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.944105 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.944387 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.944668 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.944941 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.945223 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.945493 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.945763 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.946052 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.946316 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.946529 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:04:23.946642 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.946741 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.946856 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.946965 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.947175 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.947452 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.947761 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.948034 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.948301 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.948584 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.948863 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.949157 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.949415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.949679 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.949938 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:04:23.981321 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c1e83299c0 pid: 81542 (default app) Sep 05 12:04:23.984871 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa0b08-e627-4052-9dc3-e4aa6569f283 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:23.985084 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c1e83299c0 pid: 81541 (default app) Sep 05 12:04:23.985199 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa0b08-e627-4052-9dc3-e4aa6569f283 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:23.985991 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa0b08-e627-4052-9dc3-e4aa6569f283 None None] PATH_INFO: `/v3` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:23.990452 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1/1] 10.222.0.16 () {50 vars in 791 bytes} [Mon Sep 5 12:04:23 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:04:30.956449 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf074d-3ff1-4087-bbac-60069571a74f None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:30.957034 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf074d-3ff1-4087-bbac-60069571a74f None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:30.957034 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf074d-3ff1-4087-bbac-60069571a74f None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:30.961288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1/2] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:04:30 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:04:30.967078 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5489b4f-5fd9-4c8c-ba92-20a97b2cf034 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:30.967783 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5489b4f-5fd9-4c8c-ba92-20a97b2cf034 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:30.967783 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5489b4f-5fd9-4c8c-ba92-20a97b2cf034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:31.039712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: INFO dbcounter [None req-a5489b4f-5fd9-4c8c-ba92-20a97b2cf034 None None] Registered counter for database keystone Sep 05 12:04:31.059544 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG oslo_db.sqlalchemy.engines [None req-a5489b4f-5fd9-4c8c-ba92-20a97b2cf034 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81542) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 05 12:04:31.069243 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writer thread running {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 05 12:04:31.267992 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG passlib.handlers.bcrypt [None req-a5489b4f-5fd9-4c8c-ba92-20a97b2cf034 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81542) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 05 12:04:31.268320 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG passlib.handlers.bcrypt [None req-a5489b4f-5fd9-4c8c-ba92-20a97b2cf034 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81542) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 05 12:04:31.329661 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-a5489b4f-5fd9-4c8c-ba92-20a97b2cf034 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:04:31.331313 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a5489b4f-5fd9-4c8c-ba92-20a97b2cf034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:04:31.493222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 2/3] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:04:30 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 527 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 05 12:04:31.992362 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf074d-3ff1-4087-bbac-60069571a74f None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:32.062864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: INFO dbcounter [None req-ecdf074d-3ff1-4087-bbac-60069571a74f None None] Registered counter for database keystone Sep 05 12:04:32.086053 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG oslo_db.sqlalchemy.engines [None req-ecdf074d-3ff1-4087-bbac-60069571a74f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81541) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 05 12:04:32.093783 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writer thread running {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 05 12:04:32.216929 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf074d-3ff1-4087-bbac-60069571a74f None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:32.221267 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22cf79-28b6-4c34-be4d-5c5927ea3ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:32.221898 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22cf79-28b6-4c34-be4d-5c5927ea3ace admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:32.222180 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22cf79-28b6-4c34-be4d-5c5927ea3ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:32.222456 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22cf79-28b6-4c34-be4d-5c5927ea3ace admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:32.231272 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab22cf79-28b6-4c34-be4d-5c5927ea3ace admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:32.283211 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ab22cf79-28b6-4c34-be4d-5c5927ea3ace admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 05 12:04:32.283647 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab22cf79-28b6-4c34-be4d-5c5927ea3ace admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:32.294285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 2/4] 10.222.0.16 () {58 vars in 1324 bytes} [Mon Sep 5 12:04:31 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 05 12:04:32.300652 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5489b4f-5fd9-4c8c-ba92-20a97b2cf034 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:32.316491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5489b4f-5fd9-4c8c-ba92-20a97b2cf034 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:32.321399 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389846fa-1f02-4f19-989a-f0a37766e270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:32.322587 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389846fa-1f02-4f19-989a-f0a37766e270 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:32.322872 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389846fa-1f02-4f19-989a-f0a37766e270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:32.323146 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389846fa-1f02-4f19-989a-f0a37766e270 admin admin] PATH_INFO: `/v3/projects/898d4110c367417eb23c75ef5e13bda3` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:32.324843 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389846fa-1f02-4f19-989a-f0a37766e270 admin admin] RBAC: Authorizing `identity:get_project(project_id=898d4110c367417eb23c75ef5e13bda3)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:32.377992 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: WARNING keystone.common.rbac_enforcer.enforcer [None req-389846fa-1f02-4f19-989a-f0a37766e270 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 05 12:04:32.378349 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389846fa-1f02-4f19-989a-f0a37766e270 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:32.383332 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 3/5] 10.222.0.16 () {56 vars in 1207 bytes} [Mon Sep 5 12:04:32 2022] GET /identity/v3/projects/898d4110c367417eb23c75ef5e13bda3 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:04:34.423090 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf796c3-a9b5-4dbc-9511-f2f31f2a05af None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:34.423636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf796c3-a9b5-4dbc-9511-f2f31f2a05af None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:34.423738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf796c3-a9b5-4dbc-9511-f2f31f2a05af None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:34.428313 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 3/6] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:04:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:04:34.434321 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0efef3-ac29-422d-8649-aa3aea6ae219 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:34.434847 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0efef3-ac29-422d-8649-aa3aea6ae219 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:34.434847 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0efef3-ac29-422d-8649-aa3aea6ae219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:34.456571 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-1e0efef3-ac29-422d-8649-aa3aea6ae219 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:04:34.458182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1e0efef3-ac29-422d-8649-aa3aea6ae219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:04:34.466171 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 4/7] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:04:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 05 12:04:34.736968 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf796c3-a9b5-4dbc-9511-f2f31f2a05af None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:34.749374 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf796c3-a9b5-4dbc-9511-f2f31f2a05af None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:34.754565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f5954-5e5b-44c3-a195-cbf127e1b66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:34.755190 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f5954-5e5b-44c3-a195-cbf127e1b66d admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:34.755471 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f5954-5e5b-44c3-a195-cbf127e1b66d admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:34.755784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f5954-5e5b-44c3-a195-cbf127e1b66d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:34.764455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f5954-5e5b-44c3-a195-cbf127e1b66d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:34.766310 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f5954-5e5b-44c3-a195-cbf127e1b66d admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:34.775975 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 4/8] 10.222.0.16 () {58 vars in 1324 bytes} [Mon Sep 5 12:04:34 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 05 12:04:34.782150 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0efef3-ac29-422d-8649-aa3aea6ae219 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:34.791978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0efef3-ac29-422d-8649-aa3aea6ae219 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:34.796658 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c2e2aa-c594-4bab-aa0a-e5d39db5f9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:34.797793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2e2aa-c594-4bab-aa0a-e5d39db5f9cd admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:34.798069 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2e2aa-c594-4bab-aa0a-e5d39db5f9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:34.798338 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2e2aa-c594-4bab-aa0a-e5d39db5f9cd admin admin] PATH_INFO: `/v3/users/78575c7ceaa64d1b86e8eef0293f1e3c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:34.799744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c2e2aa-c594-4bab-aa0a-e5d39db5f9cd admin admin] RBAC: Authorizing `identity:get_user(user_id=78575c7ceaa64d1b86e8eef0293f1e3c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:34.801581 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c2e2aa-c594-4bab-aa0a-e5d39db5f9cd admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:34.803193 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 5/9] 10.222.0.16 () {56 vars in 1198 bytes} [Mon Sep 5 12:04:34 2022] GET /identity/v3/users/78575c7ceaa64d1b86e8eef0293f1e3c => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:04:36.841478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcf79d-e9a2-4958-b3d6-fe8ab37fcdcb None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:36.842252 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcf79d-e9a2-4958-b3d6-fe8ab37fcdcb None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:36.842252 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcf79d-e9a2-4958-b3d6-fe8ab37fcdcb None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:36.843658 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 5/10] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:04:36.850476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b613f-02db-4345-b70c-54ad352e40b2 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:36.851178 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b613f-02db-4345-b70c-54ad352e40b2 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:36.851576 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b613f-02db-4345-b70c-54ad352e40b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:36.872325 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-209b613f-02db-4345-b70c-54ad352e40b2 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:04:36.873521 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-209b613f-02db-4345-b70c-54ad352e40b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:04:36.970504 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 6/11] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:04:37.103743 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc1295-5b87-44c3-a79d-167c0065c323 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:37.104238 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc1295-5b87-44c3-a79d-167c0065c323 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:37.104667 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc1295-5b87-44c3-a79d-167c0065c323 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:37.106941 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 6/12] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:04:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:04:37.115967 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e931ff-3615-44ec-b2dd-528e72f47e4c None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:37.116486 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e931ff-3615-44ec-b2dd-528e72f47e4c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:37.116968 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e931ff-3615-44ec-b2dd-528e72f47e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:37.147246 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-42e931ff-3615-44ec-b2dd-528e72f47e4c None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:04:37.149213 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-42e931ff-3615-44ec-b2dd-528e72f47e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:04:37.176991 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:37.240028 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc1295-5b87-44c3-a79d-167c0065c323 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:37.249978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 7/13] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:04:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:04:37.491098 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc1295-5b87-44c3-a79d-167c0065c323 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:37.517222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e931ff-3615-44ec-b2dd-528e72f47e4c None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:37.544809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924163a-398b-4b14-aff8-533e9195c0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:37.545683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924163a-398b-4b14-aff8-533e9195c0ce None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:37.545964 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924163a-398b-4b14-aff8-533e9195c0ce None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:37.546237 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924163a-398b-4b14-aff8-533e9195c0ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:37.558803 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924163a-398b-4b14-aff8-533e9195c0ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:04:37.577493 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-0924163a-398b-4b14-aff8-533e9195c0ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:04:37.577493 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:04:37.577493 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:04:37.577493 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:04:37.577493 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:04:37.577493 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:04:37.577493 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:04:37.577493 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:04:37.577493 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:04:37.577493 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:04:37.577493 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:04:37.577493 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:04:37.577493 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:04:37.577493 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:04:37.577493 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:04:37.577493 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:04:37.579377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:04:37.579377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:04:37.579377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:04:37.579377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:04:37.579377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:04:37.579377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:04:37.579377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:04:37.579377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:04:37.579377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:04:37.579377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:04:37.579377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:04:37.579377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:04:37.579377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:04:37.579377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:04:37.579377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:04:37.579377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:04:37.580552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:04:37.580552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:04:37.580552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:04:37.580552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:04:37.580552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:04:37.580552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:04:37.580552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:04:37.580552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:04:37.580552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:04:37.580552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:04:37.580552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:04:37.580552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:04:37.580552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 7/14] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:04:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:04:37.588180 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924163a-398b-4b14-aff8-533e9195c0ce None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:37.646167 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:37.681184 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e931ff-3615-44ec-b2dd-528e72f47e4c None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:37.717600 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c79e9-a635-422d-b17e-ad03a55a1fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:37.718227 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c79e9-a635-422d-b17e-ad03a55a1fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:37.718508 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c79e9-a635-422d-b17e-ad03a55a1fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:37.718783 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c79e9-a635-422d-b17e-ad03a55a1fb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:37.742198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:37.764780 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924163a-398b-4b14-aff8-533e9195c0ce None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:37.801356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c79e9-a635-422d-b17e-ad03a55a1fb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:37.803979 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c79e9-a635-422d-b17e-ad03a55a1fb2 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:37.807480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e51fb-ec95-4626-bddf-41397aed43e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:37.808127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e51fb-ec95-4626-bddf-41397aed43e2 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:37.808406 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e51fb-ec95-4626-bddf-41397aed43e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:37.808691 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e51fb-ec95-4626-bddf-41397aed43e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:37.809372 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e51fb-ec95-4626-bddf-41397aed43e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:04:37.809669 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e51fb-ec95-4626-bddf-41397aed43e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:37.811064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e51fb-ec95-4626-bddf-41397aed43e2 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:37.824786 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 8/15] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:04:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:04:37.830487 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e51fb-ec95-4626-bddf-41397aed43e2 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:37.928310 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 8/16] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:04:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 05 12:04:37.937155 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c79e9-a635-422d-b17e-ad03a55a1fb2 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:37.948796 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:37.958492 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e51fb-ec95-4626-bddf-41397aed43e2 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:38.001057 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa114eb6-9c9c-49ce-9dc7-76247b4a4a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:38.002084 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa114eb6-9c9c-49ce-9dc7-76247b4a4a99 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:38.002537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa114eb6-9c9c-49ce-9dc7-76247b4a4a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:38.002981 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa114eb6-9c9c-49ce-9dc7-76247b4a4a99 None admin] PATH_INFO: `/v3/users/78575c7ceaa64d1b86e8eef0293f1e3c` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:38.005509 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa114eb6-9c9c-49ce-9dc7-76247b4a4a99 None admin] RBAC: Authorizing `identity:get_user(user_id=78575c7ceaa64d1b86e8eef0293f1e3c)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:38.008041 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa114eb6-9c9c-49ce-9dc7-76247b4a4a99 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:38.008908 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:38.010187 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 9/17] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:04:37 2022] GET /identity/v3/users/78575c7ceaa64d1b86e8eef0293f1e3c => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:04:38.017256 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa114eb6-9c9c-49ce-9dc7-76247b4a4a99 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:38.071889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c79e9-a635-422d-b17e-ad03a55a1fb2 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:38.120740 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea241e-7e92-4c9c-999b-78260aebb28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:38.121878 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea241e-7e92-4c9c-999b-78260aebb28b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:38.122387 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea241e-7e92-4c9c-999b-78260aebb28b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:38.122867 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea241e-7e92-4c9c-999b-78260aebb28b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:38.143112 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea241e-7e92-4c9c-999b-78260aebb28b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:38.145987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea241e-7e92-4c9c-999b-78260aebb28b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:38.147097 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa114eb6-9c9c-49ce-9dc7-76247b4a4a99 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:38.181417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-3dea241e-7e92-4c9c-999b-78260aebb28b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:04:38.181417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:04:38.181417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:04:38.181417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:04:38.181417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:04:38.181417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:04:38.181417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:04:38.181417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:04:38.181417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:04:38.181417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:04:38.181417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:04:38.181417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:04:38.181417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:04:38.181417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:04:38.181417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:04:38.181417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:04:38.183132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:04:38.183132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:04:38.183132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:04:38.183132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:04:38.183132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:04:38.183132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:04:38.183132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:04:38.183132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:04:38.183132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:04:38.183132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:04:38.183132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:04:38.183132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:04:38.183132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:04:38.183132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:04:38.183132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:04:38.183132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:04:38.183132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:04:38.184795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:04:38.184795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:04:38.184795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:04:38.184795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 9/18] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:04:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:04:38.189541 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea241e-7e92-4c9c-999b-78260aebb28b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:38.196698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14872ac-a584-4e4b-852d-5f6c161a64c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:38.197794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14872ac-a584-4e4b-852d-5f6c161a64c9 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:38.198272 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14872ac-a584-4e4b-852d-5f6c161a64c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:38.198771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14872ac-a584-4e4b-852d-5f6c161a64c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:38.201999 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14872ac-a584-4e4b-852d-5f6c161a64c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:38.204931 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14872ac-a584-4e4b-852d-5f6c161a64c9 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:38.207835 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 10/19] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:04:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:04:38.213857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14872ac-a584-4e4b-852d-5f6c161a64c9 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:38.260937 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:38.328921 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea241e-7e92-4c9c-999b-78260aebb28b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:38.331783 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:38.368511 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d456a75-e0a2-462a-891b-2ddd8d6ecfcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:38.369131 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d456a75-e0a2-462a-891b-2ddd8d6ecfcb None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:38.369435 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d456a75-e0a2-462a-891b-2ddd8d6ecfcb None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:38.369697 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d456a75-e0a2-462a-891b-2ddd8d6ecfcb None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:38.370336 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d456a75-e0a2-462a-891b-2ddd8d6ecfcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:04:38.370638 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d456a75-e0a2-462a-891b-2ddd8d6ecfcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:38.372140 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d456a75-e0a2-462a-891b-2ddd8d6ecfcb None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:38.375993 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14872ac-a584-4e4b-852d-5f6c161a64c9 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:38.392303 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 10/20] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:04:38.421535 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37028ce3-ce4a-4490-b192-64d0c3abf918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:38.422300 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37028ce3-ce4a-4490-b192-64d0c3abf918 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:38.422612 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37028ce3-ce4a-4490-b192-64d0c3abf918 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:38.422925 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37028ce3-ce4a-4490-b192-64d0c3abf918 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:38.423814 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37028ce3-ce4a-4490-b192-64d0c3abf918 None admin] RBAC: Adding query filter params (role.id=618c48706f7f445abeddf7eb2f8e927a, scope.domain.id=default, user.id=78575c7ceaa64d1b86e8eef0293f1e3c) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:04:38.424161 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37028ce3-ce4a-4490-b192-64d0c3abf918 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:38.425752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37028ce3-ce4a-4490-b192-64d0c3abf918 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:38.436960 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 11/21] 10.222.0.16 () {56 vars in 1322 bytes} [Mon Sep 5 12:04:38 2022] GET /identity/v3/role_assignments?user.id=78575c7ceaa64d1b86e8eef0293f1e3c&scope.domain.id=default&role.id=618c48706f7f445abeddf7eb2f8e927a => generated 233 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:04:40.764160 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caeb0c3-ac8f-4859-ab96-4ab418a6e9cb None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:40.764884 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caeb0c3-ac8f-4859-ab96-4ab418a6e9cb None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:40.764884 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caeb0c3-ac8f-4859-ab96-4ab418a6e9cb None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:40.766014 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 11/22] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:04:40.771807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67e0e0-53dc-4a68-99dd-ccda991a776b None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:40.772692 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67e0e0-53dc-4a68-99dd-ccda991a776b None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:40.773141 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67e0e0-53dc-4a68-99dd-ccda991a776b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:40.799076 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG passlib.handlers.bcrypt [None req-ba67e0e0-53dc-4a68-99dd-ccda991a776b None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81541) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 05 12:04:40.799782 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG passlib.handlers.bcrypt [None req-ba67e0e0-53dc-4a68-99dd-ccda991a776b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81541) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 05 12:04:40.864636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-ba67e0e0-53dc-4a68-99dd-ccda991a776b None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:04:40.866674 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-ba67e0e0-53dc-4a68-99dd-ccda991a776b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:04:40.979829 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 12/23] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:04:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:04:41.169202 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918132a3-ba96-4337-bc96-229650b02dc1 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:41.169873 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918132a3-ba96-4337-bc96-229650b02dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:41.170383 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918132a3-ba96-4337-bc96-229650b02dc1 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:41.172523 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 12/24] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:04:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:04:41.178909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfd452-153b-42b9-8214-8200103ab4fa None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:41.179539 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfd452-153b-42b9-8214-8200103ab4fa None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:41.180015 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfd452-153b-42b9-8214-8200103ab4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:41.208685 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-48bfd452-153b-42b9-8214-8200103ab4fa None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:04:41.210378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-48bfd452-153b-42b9-8214-8200103ab4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:04:41.235155 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:41.291444 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918132a3-ba96-4337-bc96-229650b02dc1 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:41.318946 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 13/25] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:04:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:04:41.445623 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918132a3-ba96-4337-bc96-229650b02dc1 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:41.502050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a6b66-cae6-4d36-ac6b-56b131712cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:41.503077 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a6b66-cae6-4d36-ac6b-56b131712cc5 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:41.503584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a6b66-cae6-4d36-ac6b-56b131712cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:41.504075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a6b66-cae6-4d36-ac6b-56b131712cc5 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:41.504992 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a6b66-cae6-4d36-ac6b-56b131712cc5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:41.506615 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a6b66-cae6-4d36-ac6b-56b131712cc5 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:41.648841 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 13/26] 10.222.0.16 () {60 vars in 1131 bytes} [Mon Sep 5 12:04:41 2022] POST /identity/v3/roles => generated 219 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:04:42.003512 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de418ebe-6374-47ce-ad5b-946c0225ee78 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:42.004093 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de418ebe-6374-47ce-ad5b-946c0225ee78 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:42.004093 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de418ebe-6374-47ce-ad5b-946c0225ee78 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:42.005950 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 14/27] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:04:42.011886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8f6e4-e7c2-4613-b41c-6f63f3174299 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:42.012849 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8f6e4-e7c2-4613-b41c-6f63f3174299 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:42.013329 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8f6e4-e7c2-4613-b41c-6f63f3174299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:42.038632 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-3ca8f6e4-e7c2-4613-b41c-6f63f3174299 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:04:42.041185 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3ca8f6e4-e7c2-4613-b41c-6f63f3174299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:04:42.053205 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de418ebe-6374-47ce-ad5b-946c0225ee78 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:42.071595 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:42.157528 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 14/28] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:04:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:04:42.159088 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc28282-7229-4f95-9c6e-080fb4e88d08 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:42.159377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc28282-7229-4f95-9c6e-080fb4e88d08 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:42.159689 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc28282-7229-4f95-9c6e-080fb4e88d08 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:42.163626 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 15/29] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:04:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:04:42.168506 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a77912-51f7-45b6-8eb3-755f7f4a0a03 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:42.168779 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a77912-51f7-45b6-8eb3-755f7f4a0a03 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:42.169059 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a77912-51f7-45b6-8eb3-755f7f4a0a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:42.209119 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-22a77912-51f7-45b6-8eb3-755f7f4a0a03 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:04:42.210360 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-22a77912-51f7-45b6-8eb3-755f7f4a0a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:04:42.236844 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de418ebe-6374-47ce-ad5b-946c0225ee78 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:42.297522 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e986d3-74c0-4660-a30f-267563ad0768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:42.298230 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e986d3-74c0-4660-a30f-267563ad0768 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:42.298559 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e986d3-74c0-4660-a30f-267563ad0768 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:42.298835 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e986d3-74c0-4660-a30f-267563ad0768 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:42.328713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e986d3-74c0-4660-a30f-267563ad0768 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:42.330357 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e986d3-74c0-4660-a30f-267563ad0768 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:42.355433 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-b6e986d3-74c0-4660-a30f-267563ad0768 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:04:42.355433 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:04:42.355433 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:04:42.355433 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:04:42.355433 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:04:42.355433 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:04:42.355433 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:04:42.355433 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:04:42.355433 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:04:42.355433 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:04:42.355433 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:04:42.355433 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:04:42.355433 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:04:42.355433 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:04:42.355433 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:04:42.355433 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:04:42.356398 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:04:42.356398 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:04:42.356398 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:04:42.356398 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:04:42.356398 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:04:42.356398 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:04:42.356398 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:04:42.356398 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:04:42.356398 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:04:42.356398 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:04:42.356398 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:04:42.356398 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:04:42.356398 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:04:42.356398 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:04:42.356398 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:04:42.356398 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:04:42.356398 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:04:42.357254 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:04:42.357254 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:04:42.357254 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:04:42.357254 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 15/30] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:04:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:04:42.362565 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 16/31] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:04:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:04:42.363058 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e986d3-74c0-4660-a30f-267563ad0768 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:42.385997 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:42.509411 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e986d3-74c0-4660-a30f-267563ad0768 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:42.547821 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e8137-b7c5-4e10-9986-bb04fe8fbb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:42.548360 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e8137-b7c5-4e10-9986-bb04fe8fbb13 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:42.548639 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e8137-b7c5-4e10-9986-bb04fe8fbb13 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:42.548911 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e8137-b7c5-4e10-9986-bb04fe8fbb13 None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:42.549573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e8137-b7c5-4e10-9986-bb04fe8fbb13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:04:42.549875 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e8137-b7c5-4e10-9986-bb04fe8fbb13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:42.551282 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e8137-b7c5-4e10-9986-bb04fe8fbb13 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:42.571184 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 16/32] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:04:42.576497 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a77912-51f7-45b6-8eb3-755f7f4a0a03 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:42.667152 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e8137-b7c5-4e10-9986-bb04fe8fbb13 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:42.709801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:42.745074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a77912-51f7-45b6-8eb3-755f7f4a0a03 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:42.745765 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:42.780403 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e8137-b7c5-4e10-9986-bb04fe8fbb13 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:42.794377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e41f57-541e-4004-8f0f-6a8be1ce19c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:42.795008 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e41f57-541e-4004-8f0f-6a8be1ce19c1 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:42.795640 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e41f57-541e-4004-8f0f-6a8be1ce19c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:42.795930 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e41f57-541e-4004-8f0f-6a8be1ce19c1 None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:42.797733 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e41f57-541e-4004-8f0f-6a8be1ce19c1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:42.799769 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e41f57-541e-4004-8f0f-6a8be1ce19c1 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:42.819575 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f219a-8339-4f86-aabe-dbb37275cbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:42.820396 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f219a-8339-4f86-aabe-dbb37275cbec None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:42.820743 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f219a-8339-4f86-aabe-dbb37275cbec None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:42.821126 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f219a-8339-4f86-aabe-dbb37275cbec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:42.834751 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53f219a-8339-4f86-aabe-dbb37275cbec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:04:42.840154 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 17/33] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Sep 5 12:04:42 2022] POST /identity/v3/projects => generated 301 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:04:42.840897 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-b53f219a-8339-4f86-aabe-dbb37275cbec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:04:42.840897 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:04:42.840897 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:04:42.840897 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:04:42.840897 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:04:42.840897 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:04:42.840897 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:04:42.840897 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:04:42.840897 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:04:42.840897 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:04:42.840897 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:04:42.840897 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:04:42.840897 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:04:42.840897 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:04:42.840897 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:04:42.840897 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:04:42.841752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:04:42.841752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:04:42.841752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:04:42.841752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:04:42.841752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:04:42.841752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:04:42.841752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:04:42.841752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:04:42.841752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:04:42.841752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:04:42.841752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:04:42.841752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:04:42.841752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:04:42.841752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:04:42.841752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:04:42.841752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:04:42.842562 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:04:42.842562 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:04:42.842562 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:04:42.842562 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:04:42.842562 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:04:42.842562 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:04:42.842562 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:04:42.842562 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:04:42.842562 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:04:42.842562 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:04:42.842562 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:04:42.842562 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:04:42.847903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 17/34] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:04:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:04:42.848543 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e41f57-541e-4004-8f0f-6a8be1ce19c1 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:42.854562 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f219a-8339-4f86-aabe-dbb37275cbec None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:42.993639 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f219a-8339-4f86-aabe-dbb37275cbec None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:43.044699 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:43.054092 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e41f57-541e-4004-8f0f-6a8be1ce19c1 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:43.056394 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a49a0-a1c9-4526-9f37-bdd29b66a1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:43.058087 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a49a0-a1c9-4526-9f37-bdd29b66a1d0 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:43.058577 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a49a0-a1c9-4526-9f37-bdd29b66a1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:43.059027 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a49a0-a1c9-4526-9f37-bdd29b66a1d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:43.059893 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a49a0-a1c9-4526-9f37-bdd29b66a1d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:04:43.060348 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a49a0-a1c9-4526-9f37-bdd29b66a1d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:43.062616 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a49a0-a1c9-4526-9f37-bdd29b66a1d0 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:43.076486 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 18/35] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:04:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:04:43.077991 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a7066-95e3-4c48-b2fa-d1ae8b866de6 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:43.078977 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a7066-95e3-4c48-b2fa-d1ae8b866de6 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:43.079411 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a7066-95e3-4c48-b2fa-d1ae8b866de6 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:43.081165 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 19/36] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:04:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:04:43.086490 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a7066-95e3-4c48-b2fa-d1ae8b866de6 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:43.118108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1887f-866a-4d1d-8466-4d006eb50e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:43.119597 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1887f-866a-4d1d-8466-4d006eb50e53 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:43.120096 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1887f-866a-4d1d-8466-4d006eb50e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:43.120583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1887f-866a-4d1d-8466-4d006eb50e53 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:43.122276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b1887f-866a-4d1d-8466-4d006eb50e53 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:43.127072 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b1887f-866a-4d1d-8466-4d006eb50e53 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:43.138374 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:43.160053 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 18/37] 10.222.0.16 () {60 vars in 1131 bytes} [Mon Sep 5 12:04:42 2022] POST /identity/v3/roles => generated 225 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:04:43.165760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1691f-73b7-4280-b3da-11db9658e982 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:43.166126 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1691f-73b7-4280-b3da-11db9658e982 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:43.166479 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1691f-73b7-4280-b3da-11db9658e982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:43.192807 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-dcd1691f-73b7-4280-b3da-11db9658e982 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:04:43.194082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-dcd1691f-73b7-4280-b3da-11db9658e982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:04:43.218633 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a7066-95e3-4c48-b2fa-d1ae8b866de6 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:43.273089 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c4f3e-7266-487b-97fd-9670d2f9b141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:43.273756 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c4f3e-7266-487b-97fd-9670d2f9b141 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:43.274088 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c4f3e-7266-487b-97fd-9670d2f9b141 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:43.274347 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c4f3e-7266-487b-97fd-9670d2f9b141 None admin] PATH_INFO: `/v3/users/78575c7ceaa64d1b86e8eef0293f1e3c` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:43.275977 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160c4f3e-7266-487b-97fd-9670d2f9b141 None admin] RBAC: Authorizing `identity:get_user(user_id=78575c7ceaa64d1b86e8eef0293f1e3c)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:43.277536 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160c4f3e-7266-487b-97fd-9670d2f9b141 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:43.279892 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 20/38] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:04:43 2022] GET /identity/v3/users/78575c7ceaa64d1b86e8eef0293f1e3c => generated 247 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:04:43.281177 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a01ef6-8e0a-42d2-90b1-1a75b315dc52 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:43.282046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a01ef6-8e0a-42d2-90b1-1a75b315dc52 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:43.282283 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a01ef6-8e0a-42d2-90b1-1a75b315dc52 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:43.283874 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 21/39] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:04:43.287249 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a01ef6-8e0a-42d2-90b1-1a75b315dc52 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:43.293332 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 19/40] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:04:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:04:43.295774 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639f586-57be-4189-9774-0ff845f0ab7e None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:43.296091 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639f586-57be-4189-9774-0ff845f0ab7e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:43.296561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639f586-57be-4189-9774-0ff845f0ab7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:43.318258 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-9639f586-57be-4189-9774-0ff845f0ab7e None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:04:43.319440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9639f586-57be-4189-9774-0ff845f0ab7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:04:43.404673 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:43.464221 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 20/41] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:04:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:04:43.473978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a01ef6-8e0a-42d2-90b1-1a75b315dc52 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:43.538142 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb92616-c08c-4c3e-b00a-28d92789d816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:43.539194 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb92616-c08c-4c3e-b00a-28d92789d816 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:43.539690 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb92616-c08c-4c3e-b00a-28d92789d816 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:43.540147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb92616-c08c-4c3e-b00a-28d92789d816 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:43.541925 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb92616-c08c-4c3e-b00a-28d92789d816 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:43.544747 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb92616-c08c-4c3e-b00a-28d92789d816 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:43.547807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 22/42] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:04:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:04:43.556442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639f586-57be-4189-9774-0ff845f0ab7e None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:43.647727 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:43.691302 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb92616-c08c-4c3e-b00a-28d92789d816 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:43.701338 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639f586-57be-4189-9774-0ff845f0ab7e None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:43.744230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed6fc30-c85d-4757-9fd5-f5685dbe5e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:43.745332 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6fc30-c85d-4757-9fd5-f5685dbe5e23 None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:43.745815 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6fc30-c85d-4757-9fd5-f5685dbe5e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:43.746302 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6fc30-c85d-4757-9fd5-f5685dbe5e23 None admin] PATH_INFO: `/v3/domains/default/users/78575c7ceaa64d1b86e8eef0293f1e3c/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:43.749617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed6fc30-c85d-4757-9fd5-f5685dbe5e23 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=78575c7ceaa64d1b86e8eef0293f1e3c, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:43.752943 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed6fc30-c85d-4757-9fd5-f5685dbe5e23 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:43.775648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 21/43] 10.222.0.16 () {58 vars in 1361 bytes} [Mon Sep 5 12:04:43 2022] PUT /identity/v3/domains/default/users/78575c7ceaa64d1b86e8eef0293f1e3c/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:04:43.777038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed6fc30-c85d-4757-9fd5-f5685dbe5e23 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:43.816599 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb92616-c08c-4c3e-b00a-28d92789d816 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:43.852449 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:43.869190 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe7e80-c696-439e-951c-b776ae6ad21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:43.870424 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe7e80-c696-439e-951c-b776ae6ad21b None admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:43.870929 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe7e80-c696-439e-951c-b776ae6ad21b None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:43.871407 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe7e80-c696-439e-951c-b776ae6ad21b None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:43.872655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fe7e80-c696-439e-951c-b776ae6ad21b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:43.875769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fe7e80-c696-439e-951c-b776ae6ad21b None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:44.030937 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed6fc30-c85d-4757-9fd5-f5685dbe5e23 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:44.050684 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:44.070346 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352813b-2754-4c02-9538-d1c574123537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:44.071025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352813b-2754-4c02-9538-d1c574123537 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:44.071252 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352813b-2754-4c02-9538-d1c574123537 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:44.071543 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352813b-2754-4c02-9538-d1c574123537 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:44.072668 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352813b-2754-4c02-9538-d1c574123537 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:44.074108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352813b-2754-4c02-9538-d1c574123537 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:44.075713 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 22/44] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:04:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:04:44.081276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352813b-2754-4c02-9538-d1c574123537 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:44.116555 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 23/45] 10.222.0.16 () {60 vars in 1131 bytes} [Mon Sep 5 12:04:43 2022] POST /identity/v3/roles => generated 223 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:04:44.262129 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352813b-2754-4c02-9538-d1c574123537 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:44.313908 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599bc441-3d41-44ee-8f44-d764fd33a558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:44.314860 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599bc441-3d41-44ee-8f44-d764fd33a558 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:44.315354 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599bc441-3d41-44ee-8f44-d764fd33a558 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:44.315877 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599bc441-3d41-44ee-8f44-d764fd33a558 None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:44.317092 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599bc441-3d41-44ee-8f44-d764fd33a558 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:44.319862 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599bc441-3d41-44ee-8f44-d764fd33a558 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:44.360245 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 23/46] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Sep 5 12:04:44 2022] POST /identity/v3/projects => generated 312 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:04:45.740204 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1fdbd-d19f-499f-a4a0-299c78d8c7e9 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:45.740706 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1fdbd-d19f-499f-a4a0-299c78d8c7e9 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:45.741110 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1fdbd-d19f-499f-a4a0-299c78d8c7e9 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:45.743014 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 24/47] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:04:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:04:45.750221 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532dc85a-1a28-4c21-8c5e-b4486ad67bb9 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:45.750979 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532dc85a-1a28-4c21-8c5e-b4486ad67bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:45.750979 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532dc85a-1a28-4c21-8c5e-b4486ad67bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:45.770290 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-532dc85a-1a28-4c21-8c5e-b4486ad67bb9 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:04:45.771551 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-532dc85a-1a28-4c21-8c5e-b4486ad67bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:04:45.807655 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:45.850695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 24/48] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:04:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:04:46.116610 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1fdbd-d19f-499f-a4a0-299c78d8c7e9 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:46.262283 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1fdbd-d19f-499f-a4a0-299c78d8c7e9 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:46.310318 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95731743-ff53-4d0f-aa64-d60321be7f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:46.311046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95731743-ff53-4d0f-aa64-d60321be7f12 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:46.311379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95731743-ff53-4d0f-aa64-d60321be7f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:46.311757 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95731743-ff53-4d0f-aa64-d60321be7f12 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:46.325246 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95731743-ff53-4d0f-aa64-d60321be7f12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:04:46.330815 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-95731743-ff53-4d0f-aa64-d60321be7f12 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:04:46.330815 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:04:46.330815 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:04:46.330815 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:04:46.330815 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:04:46.330815 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:04:46.330815 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:04:46.330815 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:04:46.330815 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:04:46.330815 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:04:46.330815 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:04:46.330815 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:04:46.330815 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:04:46.330815 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:04:46.330815 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:04:46.330815 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:04:46.335995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:04:46.335995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:04:46.335995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:04:46.335995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:04:46.335995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:04:46.335995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:04:46.335995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:04:46.335995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:04:46.335995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:04:46.335995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:04:46.335995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:04:46.335995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:04:46.335995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:04:46.335995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:04:46.335995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:04:46.335995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:04:46.336909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:04:46.336909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:04:46.336909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:04:46.336909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:04:46.336909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:04:46.336909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:04:46.336909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:04:46.336909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:04:46.336909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:04:46.336909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:04:46.336909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:04:46.336909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:04:46.336909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 25/49] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:04:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:04:46.339811 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532dc85a-1a28-4c21-8c5e-b4486ad67bb9 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:46.346997 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53321ee-db16-40fc-a82a-f10cb24c1c07 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:46.347438 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53321ee-db16-40fc-a82a-f10cb24c1c07 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:46.347900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53321ee-db16-40fc-a82a-f10cb24c1c07 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:46.349397 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 26/50] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:04:46.357967 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533479bc-4e75-40f2-8974-e22d6c000488 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:46.358432 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533479bc-4e75-40f2-8974-e22d6c000488 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:46.358887 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533479bc-4e75-40f2-8974-e22d6c000488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:46.384573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-533479bc-4e75-40f2-8974-e22d6c000488 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:04:46.386401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-533479bc-4e75-40f2-8974-e22d6c000488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:04:46.436438 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:46.464060 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532dc85a-1a28-4c21-8c5e-b4486ad67bb9 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:46.466159 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 27/51] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:04:46.504705 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954ff7f-2062-4a83-bd5e-ec81f5359838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:46.505334 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954ff7f-2062-4a83-bd5e-ec81f5359838 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:46.505615 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954ff7f-2062-4a83-bd5e-ec81f5359838 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:46.505888 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954ff7f-2062-4a83-bd5e-ec81f5359838 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:46.506563 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1954ff7f-2062-4a83-bd5e-ec81f5359838 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:04:46.506870 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1954ff7f-2062-4a83-bd5e-ec81f5359838 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:46.508296 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1954ff7f-2062-4a83-bd5e-ec81f5359838 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:46.521308 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 25/52] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:04:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:04:46.527166 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533479bc-4e75-40f2-8974-e22d6c000488 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:46.630066 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533479bc-4e75-40f2-8974-e22d6c000488 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:46.667556 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f9196-f473-45e7-adef-eee0c7b5d4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:46.668342 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f9196-f473-45e7-adef-eee0c7b5d4fe None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:46.668683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f9196-f473-45e7-adef-eee0c7b5d4fe None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:46.669035 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f9196-f473-45e7-adef-eee0c7b5d4fe None admin] PATH_INFO: `/v3/users/78575c7ceaa64d1b86e8eef0293f1e3c` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:46.670284 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f9196-f473-45e7-adef-eee0c7b5d4fe None admin] RBAC: Authorizing `identity:get_user(user_id=78575c7ceaa64d1b86e8eef0293f1e3c)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:46.672597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f9196-f473-45e7-adef-eee0c7b5d4fe None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:46.674280 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 28/53] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:04:46 2022] GET /identity/v3/users/78575c7ceaa64d1b86e8eef0293f1e3c => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:04:46.680216 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954ff7f-2062-4a83-bd5e-ec81f5359838 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:46.710681 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:46.791559 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954ff7f-2062-4a83-bd5e-ec81f5359838 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:46.805055 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f9196-f473-45e7-adef-eee0c7b5d4fe None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:46.835852 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b522b2f-79f4-412a-9a72-719bb2b3a903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:46.837431 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b522b2f-79f4-412a-9a72-719bb2b3a903 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:46.838343 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b522b2f-79f4-412a-9a72-719bb2b3a903 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:46.838642 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b522b2f-79f4-412a-9a72-719bb2b3a903 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:46.840134 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b522b2f-79f4-412a-9a72-719bb2b3a903 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:46.842240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b522b2f-79f4-412a-9a72-719bb2b3a903 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:46.844227 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 26/54] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:04:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:04:46.850273 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b522b2f-79f4-412a-9a72-719bb2b3a903 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:46.857110 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:46.917183 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f9196-f473-45e7-adef-eee0c7b5d4fe None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:46.967014 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:46.974387 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b522b2f-79f4-412a-9a72-719bb2b3a903 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:47.017687 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5162853-9060-481d-a0fa-8b90bb4c3a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:47.017944 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5162853-9060-481d-a0fa-8b90bb4c3a44 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:47.018114 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5162853-9060-481d-a0fa-8b90bb4c3a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:47.018285 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5162853-9060-481d-a0fa-8b90bb4c3a44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:47.019024 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5162853-9060-481d-a0fa-8b90bb4c3a44 None admin] RBAC: Adding query filter params (role.id=618c48706f7f445abeddf7eb2f8e927a, scope.domain.id=default, user.id=78575c7ceaa64d1b86e8eef0293f1e3c) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:04:47.019331 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5162853-9060-481d-a0fa-8b90bb4c3a44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:47.020838 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5162853-9060-481d-a0fa-8b90bb4c3a44 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:47.029285 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 27/55] 10.222.0.16 () {56 vars in 1322 bytes} [Mon Sep 5 12:04:46 2022] GET /identity/v3/role_assignments?user.id=78575c7ceaa64d1b86e8eef0293f1e3c&scope.domain.id=default&role.id=618c48706f7f445abeddf7eb2f8e927a => generated 530 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:04:47.069861 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7c851-7ae6-4448-a414-8ec5617d58cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:47.070518 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7c851-7ae6-4448-a414-8ec5617d58cf None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:47.070841 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7c851-7ae6-4448-a414-8ec5617d58cf None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:47.071068 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7c851-7ae6-4448-a414-8ec5617d58cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:47.072354 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7c851-7ae6-4448-a414-8ec5617d58cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:47.073846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7c851-7ae6-4448-a414-8ec5617d58cf None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:47.075540 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 29/56] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:04:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:04:47.082330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5162853-9060-481d-a0fa-8b90bb4c3a44 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:47.209569 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5162853-9060-481d-a0fa-8b90bb4c3a44 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:47.250430 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6216a7d-f09e-4f79-8245-3ef8c730679a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:47.251064 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6216a7d-f09e-4f79-8245-3ef8c730679a None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:47.251350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6216a7d-f09e-4f79-8245-3ef8c730679a None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:47.251666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6216a7d-f09e-4f79-8245-3ef8c730679a None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:47.252367 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6216a7d-f09e-4f79-8245-3ef8c730679a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:47.253926 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6216a7d-f09e-4f79-8245-3ef8c730679a None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:47.286552 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 28/57] 10.222.0.16 () {60 vars in 1140 bytes} [Mon Sep 5 12:04:47 2022] POST /identity/v3/projects => generated 298 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:04:49.234398 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cc1f1-6cd1-4f63-a5f8-774ae00a2b7a None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:49.234923 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cc1f1-6cd1-4f63-a5f8-774ae00a2b7a None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:49.234923 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cc1f1-6cd1-4f63-a5f8-774ae00a2b7a None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:49.236313 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 30/58] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:04:49.241880 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcb41f-a88c-42ae-9873-faa97d91e932 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:49.242377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcb41f-a88c-42ae-9873-faa97d91e932 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:49.242509 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcb41f-a88c-42ae-9873-faa97d91e932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:49.260771 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-aadcb41f-a88c-42ae-9873-faa97d91e932 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:04:49.261972 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-aadcb41f-a88c-42ae-9873-faa97d91e932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:04:49.270658 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:49.332341 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 29/59] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:04:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:04:49.601212 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4cc1f1-6cd1-4f63-a5f8-774ae00a2b7a None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:49.707453 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4cc1f1-6cd1-4f63-a5f8-774ae00a2b7a None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:49.721905 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:49.747356 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5886bb8-53c0-40ba-a4e8-c8c363529396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:49.748047 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5886bb8-53c0-40ba-a4e8-c8c363529396 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:49.748331 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5886bb8-53c0-40ba-a4e8-c8c363529396 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:49.748606 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5886bb8-53c0-40ba-a4e8-c8c363529396 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:49.750327 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5886bb8-53c0-40ba-a4e8-c8c363529396 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:49.751819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5886bb8-53c0-40ba-a4e8-c8c363529396 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:49.753411 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 31/60] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:04:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:04:49.759245 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadcb41f-a88c-42ae-9873-faa97d91e932 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:49.860912 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadcb41f-a88c-42ae-9873-faa97d91e932 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:49.897085 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e63d01-3ed5-4ad4-971a-1997ccac5822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:49.897699 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e63d01-3ed5-4ad4-971a-1997ccac5822 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:49.897980 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e63d01-3ed5-4ad4-971a-1997ccac5822 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:49.898256 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e63d01-3ed5-4ad4-971a-1997ccac5822 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:49.898937 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e63d01-3ed5-4ad4-971a-1997ccac5822 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:49.900431 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e63d01-3ed5-4ad4-971a-1997ccac5822 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:49.924015 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:49.929920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 30/61] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Sep 5 12:04:49 2022] POST /identity/v3/users => generated 275 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:04:52.930135 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da082a-a111-4ca2-962c-ab282a335aea None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:52.931073 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da082a-a111-4ca2-962c-ab282a335aea None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:52.931500 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da082a-a111-4ca2-962c-ab282a335aea None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:52.933170 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 32/62] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:04:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:04:52.939566 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec3898-fb51-4d2d-8e69-fd8c8b4c3e4c None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:52.940417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec3898-fb51-4d2d-8e69-fd8c8b4c3e4c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:52.940851 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec3898-fb51-4d2d-8e69-fd8c8b4c3e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:52.968708 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-e1ec3898-fb51-4d2d-8e69-fd8c8b4c3e4c None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:04:52.970219 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e1ec3898-fb51-4d2d-8e69-fd8c8b4c3e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:04:53.060388 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 31/63] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:04:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:04:53.431964 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c137959-6827-47d4-9329-f05ae5ed2ffe None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:53.432485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c137959-6827-47d4-9329-f05ae5ed2ffe None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:53.432913 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c137959-6827-47d4-9329-f05ae5ed2ffe None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:53.434460 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 33/64] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:04:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:04:53.440061 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc1dc1-c7c2-4ce4-8d54-953b05cb9488 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:53.440560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc1dc1-c7c2-4ce4-8d54-953b05cb9488 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:53.440987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc1dc1-c7c2-4ce4-8d54-953b05cb9488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:53.466698 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-cefc1dc1-c7c2-4ce4-8d54-953b05cb9488 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:04:53.468183 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-cefc1dc1-c7c2-4ce4-8d54-953b05cb9488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:04:53.504189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c137959-6827-47d4-9329-f05ae5ed2ffe None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:53.571353 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 32/65] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:04:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:04:53.641689 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d2779-e17d-42f4-b967-cb99d5222265 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:53.642299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d2779-e17d-42f4-b967-cb99d5222265 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:53.642810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d2779-e17d-42f4-b967-cb99d5222265 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:53.644435 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 33/66] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:04:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:04:53.652733 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68c60e-4225-4030-ac7a-273f83e060c4 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:53.653332 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68c60e-4225-4030-ac7a-273f83e060c4 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:53.654058 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68c60e-4225-4030-ac7a-273f83e060c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:53.683563 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c137959-6827-47d4-9329-f05ae5ed2ffe None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:53.687490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-9e68c60e-4225-4030-ac7a-273f83e060c4 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:04:53.694529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9e68c60e-4225-4030-ac7a-273f83e060c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:04:53.759602 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f2114-f3a0-4d88-9278-76ed98695f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:53.760631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f2114-f3a0-4d88-9278-76ed98695f1c None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:53.761139 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f2114-f3a0-4d88-9278-76ed98695f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:53.761613 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f2114-f3a0-4d88-9278-76ed98695f1c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:53.790550 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f2114-f3a0-4d88-9278-76ed98695f1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:04:53.795950 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-ef2f2114-f3a0-4d88-9278-76ed98695f1c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:04:53.795950 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:04:53.795950 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:04:53.795950 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:04:53.795950 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:04:53.795950 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:04:53.795950 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:04:53.795950 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:04:53.795950 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:04:53.795950 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:04:53.795950 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:04:53.795950 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:04:53.795950 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:04:53.795950 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:04:53.795950 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:04:53.795950 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:04:53.796903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:04:53.796903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:04:53.796903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:04:53.796903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:04:53.796903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:04:53.796903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:04:53.796903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:04:53.796903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:04:53.796903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:04:53.796903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:04:53.796903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:04:53.796903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:04:53.796903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:04:53.796903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:04:53.796903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:04:53.796903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:04:53.797683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:04:53.797683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:04:53.797683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:04:53.797683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:04:53.797683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:04:53.797683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:04:53.797683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:04:53.797683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:04:53.797683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:04:53.797683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:04:53.797683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:04:53.797683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:04:53.804162 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 34/67] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:04:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:04:53.810766 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f2114-f3a0-4d88-9278-76ed98695f1c None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:53.823287 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:53.858225 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 34/68] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:04:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:04:53.863612 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68c60e-4225-4030-ac7a-273f83e060c4 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:53.888090 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:53.965947 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f2114-f3a0-4d88-9278-76ed98695f1c None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:54.007511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012cfced-ca58-4aff-8a1a-e5dc99a1b3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:54.008333 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012cfced-ca58-4aff-8a1a-e5dc99a1b3e3 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:54.008711 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012cfced-ca58-4aff-8a1a-e5dc99a1b3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:54.009094 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012cfced-ca58-4aff-8a1a-e5dc99a1b3e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:54.010431 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012cfced-ca58-4aff-8a1a-e5dc99a1b3e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:04:54.010816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012cfced-ca58-4aff-8a1a-e5dc99a1b3e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:54.012255 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012cfced-ca58-4aff-8a1a-e5dc99a1b3e3 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:54.023769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 35/69] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Sep 5 12:04:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:04:54.024725 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68c60e-4225-4030-ac7a-273f83e060c4 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:54.029199 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cdd4e3-5158-461b-9247-b85b34ce2d3d None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:54.029478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cdd4e3-5158-461b-9247-b85b34ce2d3d None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:54.029749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cdd4e3-5158-461b-9247-b85b34ce2d3d None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:54.031139 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 36/70] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:04:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:04:54.036394 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36375f-a596-433a-94df-1f980bfbb0be None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:54.036680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36375f-a596-433a-94df-1f980bfbb0be None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:54.036958 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36375f-a596-433a-94df-1f980bfbb0be None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:54.038568 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 37/71] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:04:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:04:54.039158 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36375f-a596-433a-94df-1f980bfbb0be None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:54.076380 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f2273-ed27-4546-ac36-be54b629c9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:54.077104 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f2273-ed27-4546-ac36-be54b629c9b9 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:54.077640 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f2273-ed27-4546-ac36-be54b629c9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:54.078724 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f2273-ed27-4546-ac36-be54b629c9b9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:54.096872 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f2273-ed27-4546-ac36-be54b629c9b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:04:54.103476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-483f2273-ed27-4546-ac36-be54b629c9b9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:04:54.103476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:04:54.103476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:04:54.103476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:04:54.103476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:04:54.103476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:04:54.103476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:04:54.103476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:04:54.103476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:04:54.103476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:04:54.103476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:04:54.103476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:04:54.103476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:04:54.103476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:04:54.103476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:04:54.103476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:04:54.105194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:04:54.105194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:04:54.105194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:04:54.105194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:04:54.105194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:04:54.105194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:04:54.105194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:04:54.105194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:04:54.105194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:04:54.105194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:04:54.105194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:04:54.105194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:04:54.105194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:04:54.105194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:04:54.105194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:04:54.105194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:04:54.106688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:04:54.106688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:04:54.106688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:04:54.106688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:04:54.106688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:04:54.106688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:04:54.106688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:04:54.106688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:04:54.106688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:04:54.106688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:04:54.106688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:04:54.106688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:04:54.106688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 35/72] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Sep 5 12:04:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:04:54.106688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b0ab9-dcba-4240-a0a2-7a0beb126769 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:54.109581 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b0ab9-dcba-4240-a0a2-7a0beb126769 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:54.109581 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b0ab9-dcba-4240-a0a2-7a0beb126769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:54.129163 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-8d6b0ab9-dcba-4240-a0a2-7a0beb126769 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:04:54.130360 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8d6b0ab9-dcba-4240-a0a2-7a0beb126769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:04:54.186453 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36375f-a596-433a-94df-1f980bfbb0be None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:54.188778 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:54.209487 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:54.226837 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa7dec-8322-496f-abe0-da9cf320c727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:54.227380 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa7dec-8322-496f-abe0-da9cf320c727 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:54.227698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa7dec-8322-496f-abe0-da9cf320c727 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:54.227981 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa7dec-8322-496f-abe0-da9cf320c727 None admin] PATH_INFO: `/v3/users/10881ece7d184527a88b151aac434c0c` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:54.251562 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 36/73] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:04:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:04:54.251562 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c61170-3c54-4d52-8fe6-d3833d399f21 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:54.251562 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c61170-3c54-4d52-8fe6-d3833d399f21 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:54.251562 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c61170-3c54-4d52-8fe6-d3833d399f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:54.255410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa7dec-8322-496f-abe0-da9cf320c727 None admin] RBAC: Authorizing `identity:get_user(user_id=10881ece7d184527a88b151aac434c0c)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:54.258114 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa7dec-8322-496f-abe0-da9cf320c727 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:54.259059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 38/74] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:04:54 2022] GET /identity/v3/users/10881ece7d184527a88b151aac434c0c => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:04:54.264800 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa7dec-8322-496f-abe0-da9cf320c727 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:54.265613 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-13c61170-3c54-4d52-8fe6-d3833d399f21 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:04:54.267066 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-13c61170-3c54-4d52-8fe6-d3833d399f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:04:54.374107 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 37/75] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:04:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:04:54.375061 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c61170-3c54-4d52-8fe6-d3833d399f21 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:54.401685 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa7dec-8322-496f-abe0-da9cf320c727 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:54.458894 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fa26f-5ddb-4d60-9784-172586dd3b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:54.459758 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fa26f-5ddb-4d60-9784-172586dd3b8b None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:54.460203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fa26f-5ddb-4d60-9784-172586dd3b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:54.460749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fa26f-5ddb-4d60-9784-172586dd3b8b None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:54.461904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4fa26f-5ddb-4d60-9784-172586dd3b8b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:04:54.462371 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4fa26f-5ddb-4d60-9784-172586dd3b8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:54.464580 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4fa26f-5ddb-4d60-9784-172586dd3b8b None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:54.486312 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 39/76] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:04:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:04:54.488339 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fa26f-5ddb-4d60-9784-172586dd3b8b None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:54.525877 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c61170-3c54-4d52-8fe6-d3833d399f21 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:54.544186 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:54.544534 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:54.597823 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329be9b2-82ed-4f7a-8c34-9e9af7df8b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:54.598921 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329be9b2-82ed-4f7a-8c34-9e9af7df8b58 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:54.599658 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329be9b2-82ed-4f7a-8c34-9e9af7df8b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:54.600162 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329be9b2-82ed-4f7a-8c34-9e9af7df8b58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:54.617878 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329be9b2-82ed-4f7a-8c34-9e9af7df8b58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:04:54.627205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-329be9b2-82ed-4f7a-8c34-9e9af7df8b58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:04:54.627205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:04:54.627205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:04:54.627205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:04:54.627205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:04:54.627205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:04:54.627205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:04:54.627205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:04:54.627205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:04:54.627205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:04:54.627205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:04:54.627205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:04:54.627205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:04:54.627205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:04:54.627205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:04:54.627205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:04:54.628699 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:04:54.628699 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:04:54.628699 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:04:54.628699 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:04:54.628699 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:04:54.628699 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:04:54.628699 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:04:54.628699 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:04:54.628699 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:04:54.628699 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:04:54.628699 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:04:54.628699 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:04:54.628699 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:04:54.628699 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:04:54.628699 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:04:54.628699 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:04:54.629782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:04:54.629782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:04:54.629782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:04:54.629782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:04:54.629782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:04:54.629782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:04:54.629782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:04:54.629782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:04:54.629782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:04:54.629782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:04:54.629782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:04:54.629782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:04:54.629782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 38/77] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:04:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:04:54.631960 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329be9b2-82ed-4f7a-8c34-9e9af7df8b58 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:54.672706 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fa26f-5ddb-4d60-9784-172586dd3b8b None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:54.724735 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b737f01-6211-456a-bebc-004334ca74ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:54.725349 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b737f01-6211-456a-bebc-004334ca74ca None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:54.725627 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b737f01-6211-456a-bebc-004334ca74ca None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:54.725896 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b737f01-6211-456a-bebc-004334ca74ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:54.761955 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329be9b2-82ed-4f7a-8c34-9e9af7df8b58 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:54.802802 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4428acdf-3f8a-4968-a980-468abf51ae6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:54.803451 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4428acdf-3f8a-4968-a980-468abf51ae6e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:54.803768 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4428acdf-3f8a-4968-a980-468abf51ae6e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:54.804057 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4428acdf-3f8a-4968-a980-468abf51ae6e None admin] PATH_INFO: `/v3/users/10881ece7d184527a88b151aac434c0c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:54.805274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4428acdf-3f8a-4968-a980-468abf51ae6e None admin] RBAC: Authorizing `identity:get_user(user_id=10881ece7d184527a88b151aac434c0c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:54.806764 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4428acdf-3f8a-4968-a980-468abf51ae6e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:54.808600 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 39/78] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:04:54 2022] GET /identity/v3/users/10881ece7d184527a88b151aac434c0c => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:04:54.809228 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4428acdf-3f8a-4968-a980-468abf51ae6e None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:54.831884 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b737f01-6211-456a-bebc-004334ca74ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:54.834611 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b737f01-6211-456a-bebc-004334ca74ca None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:54.849843 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:54.865406 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:54.931411 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4428acdf-3f8a-4968-a980-468abf51ae6e None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:54.953588 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 40/79] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:04:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:04:54.954483 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b737f01-6211-456a-bebc-004334ca74ca None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:54.973352 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc44afa-6297-4cee-a8c9-f2ad0bace980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:54.973998 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc44afa-6297-4cee-a8c9-f2ad0bace980 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:54.974283 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc44afa-6297-4cee-a8c9-f2ad0bace980 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:54.974549 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc44afa-6297-4cee-a8c9-f2ad0bace980 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:54.986375 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc44afa-6297-4cee-a8c9-f2ad0bace980 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:04:54.991761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-adc44afa-6297-4cee-a8c9-f2ad0bace980 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:04:54.991761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:04:54.991761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:04:54.991761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:04:54.991761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:04:54.991761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:04:54.991761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:04:54.991761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:04:54.991761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:04:54.991761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:04:54.991761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:04:54.991761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:04:54.991761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:04:54.991761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:04:54.991761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:04:54.991761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:04:54.993512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:04:54.993512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:04:54.993512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:04:54.993512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:04:54.993512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:04:54.993512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:04:54.993512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:04:54.993512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:04:54.993512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:04:54.993512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:04:54.993512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:04:54.993512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:04:54.993512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:04:54.993512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:04:54.993512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:04:54.993512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:04:54.995648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:04:54.995648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:04:54.995648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:04:54.995648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:04:54.995648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:04:54.995648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:04:54.995648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:04:54.995648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:04:54.995648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:04:54.995648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:04:54.995648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:04:54.995648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:04:54.995648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 40/80] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:04:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:04:54.995648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc44afa-6297-4cee-a8c9-f2ad0bace980 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:55.093384 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b737f01-6211-456a-bebc-004334ca74ca None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:55.108073 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:55.115705 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc44afa-6297-4cee-a8c9-f2ad0bace980 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:55.133114 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:55.152569 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6172d71-fa6b-4cb4-b6d9-5eb98181aa80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:55.152821 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983993f-77e9-4c18-8352-fcc74d097186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:55.153695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983993f-77e9-4c18-8352-fcc74d097186 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:55.153849 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6172d71-fa6b-4cb4-b6d9-5eb98181aa80 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:55.153978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983993f-77e9-4c18-8352-fcc74d097186 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:55.154250 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983993f-77e9-4c18-8352-fcc74d097186 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:55.154347 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6172d71-fa6b-4cb4-b6d9-5eb98181aa80 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:55.154849 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6172d71-fa6b-4cb4-b6d9-5eb98181aa80 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:55.155417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3983993f-77e9-4c18-8352-fcc74d097186 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:55.156138 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6172d71-fa6b-4cb4-b6d9-5eb98181aa80 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:04:55.156700 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6172d71-fa6b-4cb4-b6d9-5eb98181aa80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:55.156931 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3983993f-77e9-4c18-8352-fcc74d097186 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:55.158643 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 41/81] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:04:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:04:55.158915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6172d71-fa6b-4cb4-b6d9-5eb98181aa80 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:55.159468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983993f-77e9-4c18-8352-fcc74d097186 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:55.179331 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 41/82] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:04:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:04:55.179331 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6172d71-fa6b-4cb4-b6d9-5eb98181aa80 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:55.275772 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983993f-77e9-4c18-8352-fcc74d097186 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:55.311361 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6172d71-fa6b-4cb4-b6d9-5eb98181aa80 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:55.314850 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992b22f-011a-4834-b009-0b990bd087af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:55.315472 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992b22f-011a-4834-b009-0b990bd087af None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:55.315789 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992b22f-011a-4834-b009-0b990bd087af None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:55.316068 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992b22f-011a-4834-b009-0b990bd087af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:55.349371 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d5d6d-f290-4c94-8e09-5ccf8282a7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:55.349937 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d5d6d-f290-4c94-8e09-5ccf8282a7f2 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:55.350224 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d5d6d-f290-4c94-8e09-5ccf8282a7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:55.350503 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d5d6d-f290-4c94-8e09-5ccf8282a7f2 None admin] PATH_INFO: `/v3/projects/405d2cd2fe68456ca034a57f24f6b015` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:55.351744 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d5d6d-f290-4c94-8e09-5ccf8282a7f2 None admin] RBAC: Authorizing `identity:get_project(project_id=405d2cd2fe68456ca034a57f24f6b015)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:55.353209 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d5d6d-f290-4c94-8e09-5ccf8282a7f2 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:55.355105 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 42/83] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:04:55 2022] GET /identity/v3/projects/405d2cd2fe68456ca034a57f24f6b015 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:04:55.355731 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d5d6d-f290-4c94-8e09-5ccf8282a7f2 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:55.368823 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:55.415031 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992b22f-011a-4834-b009-0b990bd087af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:55.416739 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992b22f-011a-4834-b009-0b990bd087af None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:55.425941 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:55.488020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d5d6d-f290-4c94-8e09-5ccf8282a7f2 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:55.528405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d49c94-9817-483f-8561-71c46a557fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:55.529327 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d49c94-9817-483f-8561-71c46a557fce None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:55.531173 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d49c94-9817-483f-8561-71c46a557fce None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:55.531462 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d49c94-9817-483f-8561-71c46a557fce None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:55.532198 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d49c94-9817-483f-8561-71c46a557fce None admin] RBAC: Adding query filter params (name=member) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:04:55.532507 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d49c94-9817-483f-8561-71c46a557fce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:55.533925 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d49c94-9817-483f-8561-71c46a557fce None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:55.543084 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 43/84] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Sep 5 12:04:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:04:55.543983 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d49c94-9817-483f-8561-71c46a557fce None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:55.565403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 42/85] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:04:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:04:55.565403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992b22f-011a-4834-b009-0b990bd087af None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:55.629651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:55.675390 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992b22f-011a-4834-b009-0b990bd087af None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:55.678166 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d49c94-9817-483f-8561-71c46a557fce None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:55.696319 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:55.718811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a9809-97ba-4071-b187-250af630d2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:55.719242 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7329ec-d588-412e-bc94-2fb58d6b6e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:55.719441 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a9809-97ba-4071-b187-250af630d2b7 None admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:55.719756 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a9809-97ba-4071-b187-250af630d2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:55.720037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a9809-97ba-4071-b187-250af630d2b7 None admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:55.720350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7329ec-d588-412e-bc94-2fb58d6b6e8d None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:55.720723 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15a9809-97ba-4071-b187-250af630d2b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:55.720855 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7329ec-d588-412e-bc94-2fb58d6b6e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:55.721334 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7329ec-d588-412e-bc94-2fb58d6b6e8d None admin] PATH_INFO: `/v3/users/78575c7ceaa64d1b86e8eef0293f1e3c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:55.722188 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15a9809-97ba-4071-b187-250af630d2b7 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:55.723381 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7329ec-d588-412e-bc94-2fb58d6b6e8d None admin] RBAC: Authorizing `identity:get_user(user_id=78575c7ceaa64d1b86e8eef0293f1e3c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:55.726199 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7329ec-d588-412e-bc94-2fb58d6b6e8d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:55.733981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 43/86] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:04:55 2022] GET /identity/v3/users/78575c7ceaa64d1b86e8eef0293f1e3c => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:04:55.733981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7329ec-d588-412e-bc94-2fb58d6b6e8d None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:55.763557 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 44/87] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Sep 5 12:04:55 2022] POST /identity/v3/users => generated 289 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:04:55.763557 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a9809-97ba-4071-b187-250af630d2b7 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:55.885807 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7329ec-d588-412e-bc94-2fb58d6b6e8d None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:55.925227 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:55.932269 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a9809-97ba-4071-b187-250af630d2b7 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:55.934548 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f13f8-e1e9-4ad8-89a4-d09d606e72bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:55.935735 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f13f8-e1e9-4ad8-89a4-d09d606e72bf None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:55.935845 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f13f8-e1e9-4ad8-89a4-d09d606e72bf None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:55.936052 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f13f8-e1e9-4ad8-89a4-d09d606e72bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:55.937879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f13f8-e1e9-4ad8-89a4-d09d606e72bf None admin] RBAC: Adding query filter params (role.id=ede2986592354e9888dde3123b2d8c82, scope.project.id=405d2cd2fe68456ca034a57f24f6b015, user.id=10881ece7d184527a88b151aac434c0c) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:04:55.937879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f13f8-e1e9-4ad8-89a4-d09d606e72bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:55.938756 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f13f8-e1e9-4ad8-89a4-d09d606e72bf None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:55.945589 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 44/88] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:04:55 2022] GET /identity/v3/role_assignments?user.id=10881ece7d184527a88b151aac434c0c&scope.project.id=405d2cd2fe68456ca034a57f24f6b015&role.id=ede2986592354e9888dde3123b2d8c82 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:04:55.946587 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f13f8-e1e9-4ad8-89a4-d09d606e72bf None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:55.998867 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18160c-7731-4fc8-8e30-69b088ec3aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:56.000035 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18160c-7731-4fc8-8e30-69b088ec3aea None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:56.000840 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18160c-7731-4fc8-8e30-69b088ec3aea None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:56.001476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18160c-7731-4fc8-8e30-69b088ec3aea None admin] PATH_INFO: `/v3/users/10881ece7d184527a88b151aac434c0c` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:56.004294 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba18160c-7731-4fc8-8e30-69b088ec3aea None admin] RBAC: Authorizing `identity:get_user(user_id=10881ece7d184527a88b151aac434c0c)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:56.008851 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba18160c-7731-4fc8-8e30-69b088ec3aea None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:56.010913 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 45/89] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:04:55 2022] GET /identity/v3/users/10881ece7d184527a88b151aac434c0c => generated 275 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:04:56.015601 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18160c-7731-4fc8-8e30-69b088ec3aea None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:56.082630 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:56.143329 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f13f8-e1e9-4ad8-89a4-d09d606e72bf None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:56.177302 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18160c-7731-4fc8-8e30-69b088ec3aea None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:56.188245 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720604b-74a0-4bd6-a99f-af53746424ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:56.189239 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720604b-74a0-4bd6-a99f-af53746424ab None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:56.189713 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720604b-74a0-4bd6-a99f-af53746424ab None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:56.189993 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720604b-74a0-4bd6-a99f-af53746424ab None admin] PATH_INFO: `/v3/projects/405d2cd2fe68456ca034a57f24f6b015` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:56.191243 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7720604b-74a0-4bd6-a99f-af53746424ab None admin] RBAC: Authorizing `identity:get_project(project_id=405d2cd2fe68456ca034a57f24f6b015)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:56.192773 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7720604b-74a0-4bd6-a99f-af53746424ab None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:56.194485 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 45/90] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:04:55 2022] GET /identity/v3/projects/405d2cd2fe68456ca034a57f24f6b015 => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:04:56.195285 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720604b-74a0-4bd6-a99f-af53746424ab None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:56.226231 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a354ef-bc45-4b5f-85a3-328a72365bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:56.226869 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a354ef-bc45-4b5f-85a3-328a72365bdb None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:56.227151 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a354ef-bc45-4b5f-85a3-328a72365bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:56.227422 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a354ef-bc45-4b5f-85a3-328a72365bdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:56.272513 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:56.313546 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a354ef-bc45-4b5f-85a3-328a72365bdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:56.315101 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a354ef-bc45-4b5f-85a3-328a72365bdb None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:56.319393 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720604b-74a0-4bd6-a99f-af53746424ab None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:56.363228 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:56.372442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a970518-0fc4-4d5b-a6e5-fbae41e1e8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:56.373136 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a970518-0fc4-4d5b-a6e5-fbae41e1e8e4 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:56.373136 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a970518-0fc4-4d5b-a6e5-fbae41e1e8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:56.373136 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a970518-0fc4-4d5b-a6e5-fbae41e1e8e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:56.441549 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 46/91] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:04:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:04:56.442615 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a354ef-bc45-4b5f-85a3-328a72365bdb None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:56.554899 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a354ef-bc45-4b5f-85a3-328a72365bdb None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:56.561760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a970518-0fc4-4d5b-a6e5-fbae41e1e8e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:56.565196 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a970518-0fc4-4d5b-a6e5-fbae41e1e8e4 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:56.593462 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da731b-b959-43e4-95be-94077af24b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:56.594102 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da731b-b959-43e4-95be-94077af24b00 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:56.594383 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da731b-b959-43e4-95be-94077af24b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:56.594855 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da731b-b959-43e4-95be-94077af24b00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:56.596768 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da731b-b959-43e4-95be-94077af24b00 None admin] RBAC: Adding query filter params (role.id=20bf8c77837d43b590a2bd9a67d01f84, scope.project.id=405d2cd2fe68456ca034a57f24f6b015, user.id=10881ece7d184527a88b151aac434c0c) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:04:56.597333 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da731b-b959-43e4-95be-94077af24b00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:56.598857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da731b-b959-43e4-95be-94077af24b00 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:56.605385 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 47/92] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:04:56 2022] GET /identity/v3/role_assignments?user.id=10881ece7d184527a88b151aac434c0c&scope.project.id=405d2cd2fe68456ca034a57f24f6b015&role.id=20bf8c77837d43b590a2bd9a67d01f84 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:04:56.606377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da731b-b959-43e4-95be-94077af24b00 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:56.616089 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:56.721874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:56.791354 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 46/93] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:04:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:04:56.798607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a970518-0fc4-4d5b-a6e5-fbae41e1e8e4 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:56.840047 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da731b-b959-43e4-95be-94077af24b00 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:56.913371 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeae303-c002-4810-9786-f898b9a3e644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:56.913819 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a970518-0fc4-4d5b-a6e5-fbae41e1e8e4 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:56.914039 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeae303-c002-4810-9786-f898b9a3e644 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:56.914328 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeae303-c002-4810-9786-f898b9a3e644 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:56.914610 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeae303-c002-4810-9786-f898b9a3e644 None admin] PATH_INFO: `/v3/projects/405d2cd2fe68456ca034a57f24f6b015` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:56.916034 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeae303-c002-4810-9786-f898b9a3e644 None admin] RBAC: Authorizing `identity:get_project(project_id=405d2cd2fe68456ca034a57f24f6b015)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:56.917638 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeae303-c002-4810-9786-f898b9a3e644 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:56.919400 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 48/94] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:04:56 2022] GET /identity/v3/projects/405d2cd2fe68456ca034a57f24f6b015 => generated 298 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:04:56.929203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeae303-c002-4810-9786-f898b9a3e644 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:56.953621 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed404a5-1d7f-4832-b40f-a758aa8e3829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:56.954273 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed404a5-1d7f-4832-b40f-a758aa8e3829 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:56.954569 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed404a5-1d7f-4832-b40f-a758aa8e3829 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:56.954846 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed404a5-1d7f-4832-b40f-a758aa8e3829 None admin] PATH_INFO: `/v3/projects/644903936a194df991e9309a715f6877` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:56.956038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed404a5-1d7f-4832-b40f-a758aa8e3829 None admin] RBAC: Authorizing `identity:get_project(project_id=644903936a194df991e9309a715f6877)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:56.957607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed404a5-1d7f-4832-b40f-a758aa8e3829 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:56.959655 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 47/95] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:04:56 2022] GET /identity/v3/projects/644903936a194df991e9309a715f6877 => generated 312 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:04:56.965575 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed404a5-1d7f-4832-b40f-a758aa8e3829 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:04:57.001216 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:57.071592 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:57.140474 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeae303-c002-4810-9786-f898b9a3e644 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:57.148327 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed404a5-1d7f-4832-b40f-a758aa8e3829 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:04:57.209635 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea89ae9-2ff9-467f-b652-7eade7f38da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:57.210433 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea89ae9-2ff9-467f-b652-7eade7f38da6 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:57.210560 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea89ae9-2ff9-467f-b652-7eade7f38da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:57.210909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea89ae9-2ff9-467f-b652-7eade7f38da6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:57.212279 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea89ae9-2ff9-467f-b652-7eade7f38da6 None admin] RBAC: Adding query filter params (role.id=618c48706f7f445abeddf7eb2f8e927a, scope.project.id=405d2cd2fe68456ca034a57f24f6b015, user.id=78575c7ceaa64d1b86e8eef0293f1e3c) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:04:57.212598 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea89ae9-2ff9-467f-b652-7eade7f38da6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:57.214071 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea89ae9-2ff9-467f-b652-7eade7f38da6 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:57.220847 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e657e0d0-0dcd-4168-ad8d-b365298fc8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:04:57.221537 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e657e0d0-0dcd-4168-ad8d-b365298fc8d8 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:57.221826 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e657e0d0-0dcd-4168-ad8d-b365298fc8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:57.222112 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e657e0d0-0dcd-4168-ad8d-b365298fc8d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:57.223318 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e657e0d0-0dcd-4168-ad8d-b365298fc8d8 None admin] RBAC: Adding query filter params (role.id=ede2986592354e9888dde3123b2d8c82, scope.project.id=644903936a194df991e9309a715f6877, user.id=10881ece7d184527a88b151aac434c0c) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:04:57.223318 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e657e0d0-0dcd-4168-ad8d-b365298fc8d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:04:57.224828 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e657e0d0-0dcd-4168-ad8d-b365298fc8d8 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:04:57.231793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 48/96] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:04:56 2022] GET /identity/v3/role_assignments?user.id=10881ece7d184527a88b151aac434c0c&scope.project.id=644903936a194df991e9309a715f6877&role.id=ede2986592354e9888dde3123b2d8c82 => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:04:57.235271 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 49/97] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:04:56 2022] GET /identity/v3/role_assignments?user.id=78575c7ceaa64d1b86e8eef0293f1e3c&scope.project.id=405d2cd2fe68456ca034a57f24f6b015&role.id=618c48706f7f445abeddf7eb2f8e927a => generated 259 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:04:59.335216 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252a83b-deef-4ca6-88b0-ae39cffddb72 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:59.336194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252a83b-deef-4ca6-88b0-ae39cffddb72 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:59.336194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252a83b-deef-4ca6-88b0-ae39cffddb72 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:59.337476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 49/98] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:04:59.346196 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cc2d5-e649-4b55-b504-1e90d1838834 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:59.346735 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cc2d5-e649-4b55-b504-1e90d1838834 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:59.347055 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cc2d5-e649-4b55-b504-1e90d1838834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:59.409362 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-bc8cc2d5-e649-4b55-b504-1e90d1838834 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:04:59.411455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-bc8cc2d5-e649-4b55-b504-1e90d1838834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:04:59.558233 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0d5e9-8bd3-44a7-9d0a-5cc11012bddd None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:59.558936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0d5e9-8bd3-44a7-9d0a-5cc11012bddd None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:59.559440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0d5e9-8bd3-44a7-9d0a-5cc11012bddd None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:59.561310 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 50/99] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:04:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:04:59.567820 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2566dad-bfd5-470c-b86d-5a5e17589c41 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:59.568458 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2566dad-bfd5-470c-b86d-5a5e17589c41 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:59.568955 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2566dad-bfd5-470c-b86d-5a5e17589c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:59.578393 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 50/100] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:04:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:04:59.591112 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7502e2e-e12e-4af9-8ab9-8e844a9f1c0e None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:59.591563 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7502e2e-e12e-4af9-8ab9-8e844a9f1c0e None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:59.592070 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7502e2e-e12e-4af9-8ab9-8e844a9f1c0e None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:59.596686 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 51/101] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:04:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:04:59.614767 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67f906-3af4-4cb0-9645-3e84819108ec None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:04:59.614767 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67f906-3af4-4cb0-9645-3e84819108ec None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:04:59.614767 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67f906-3af4-4cb0-9645-3e84819108ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:04:59.625266 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-a2566dad-bfd5-470c-b86d-5a5e17589c41 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:04:59.628439 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a2566dad-bfd5-470c-b86d-5a5e17589c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:04:59.664893 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-fa67f906-3af4-4cb0-9645-3e84819108ec None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:04:59.666628 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-fa67f906-3af4-4cb0-9645-3e84819108ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:04:59.881891 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 51/102] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:04:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:04:59.923787 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:04:59.949765 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 52/103] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:04:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:00.034868 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2566dad-bfd5-470c-b86d-5a5e17589c41 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:00.099799 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:00.269717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2566dad-bfd5-470c-b86d-5a5e17589c41 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:00.335459 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222f00c-2f95-42d8-a09e-8a1f9d0c49f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:00.336957 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222f00c-2f95-42d8-a09e-8a1f9d0c49f9 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:00.337441 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222f00c-2f95-42d8-a09e-8a1f9d0c49f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:00.337928 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222f00c-2f95-42d8-a09e-8a1f9d0c49f9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:00.368565 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222f00c-2f95-42d8-a09e-8a1f9d0c49f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:00.373834 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-a222f00c-2f95-42d8-a09e-8a1f9d0c49f9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:05:00.373834 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:00.373834 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:00.373834 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:00.373834 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:00.373834 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:00.373834 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:00.373834 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:00.373834 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:00.373834 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:00.373834 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:00.373834 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:00.373834 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:00.373834 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:00.373834 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:00.373834 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:00.375449 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:00.375449 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:00.375449 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:00.375449 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:00.375449 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:00.375449 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:00.375449 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:00.375449 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:00.375449 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:00.375449 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:00.375449 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:00.375449 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:00.375449 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:00.375449 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:00.375449 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:00.375449 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:00.376759 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:00.376759 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:00.376759 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:00.376759 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:00.376759 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:00.376759 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:00.376759 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:00.376759 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:00.376759 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:00.376759 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:00.376759 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:05:00.376759 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:00.379088 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 52/104] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:00.388665 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67f906-3af4-4cb0-9645-3e84819108ec None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:00.455385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222f00c-2f95-42d8-a09e-8a1f9d0c49f9 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:00.721740 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67f906-3af4-4cb0-9645-3e84819108ec None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:00.747478 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:00.777070 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222f00c-2f95-42d8-a09e-8a1f9d0c49f9 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:00.844916 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e71d645-1232-4fa0-863d-3b07cf8838a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:00.846093 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0dd31b-93b7-4471-a4ef-60a9834641e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:00.846405 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71d645-1232-4fa0-863d-3b07cf8838a5 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:00.846725 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0dd31b-93b7-4471-a4ef-60a9834641e4 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:00.847012 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0dd31b-93b7-4471-a4ef-60a9834641e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:00.847294 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0dd31b-93b7-4471-a4ef-60a9834641e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:00.847736 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71d645-1232-4fa0-863d-3b07cf8838a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:00.848440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71d645-1232-4fa0-863d-3b07cf8838a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:00.852251 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0dd31b-93b7-4471-a4ef-60a9834641e4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:00.852788 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0dd31b-93b7-4471-a4ef-60a9834641e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:00.854418 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0dd31b-93b7-4471-a4ef-60a9834641e4 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:00.855395 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e71d645-1232-4fa0-863d-3b07cf8838a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:00.857028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e71d645-1232-4fa0-863d-3b07cf8838a5 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:00.861203 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 53/105] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:05:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:00.864331 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e71d645-1232-4fa0-863d-3b07cf8838a5 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:00.873193 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 53/106] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Sep 5 12:05:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 485 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:00.874718 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c296ba8-595d-4fa4-bec7-38d33a33f02c None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:00.875168 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c296ba8-595d-4fa4-bec7-38d33a33f02c None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:00.875631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c296ba8-595d-4fa4-bec7-38d33a33f02c None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:00.877939 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 54/107] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:00.878991 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c296ba8-595d-4fa4-bec7-38d33a33f02c None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:00.946133 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:01.001260 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c296ba8-595d-4fa4-bec7-38d33a33f02c None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:01.008973 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e71d645-1232-4fa0-863d-3b07cf8838a5 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:01.041084 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d7b70-f2b6-44f2-9854-ca8cda69ef1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:01.041998 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d7b70-f2b6-44f2-9854-ca8cda69ef1e None admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:01.042287 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d7b70-f2b6-44f2-9854-ca8cda69ef1e None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:01.042567 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d7b70-f2b6-44f2-9854-ca8cda69ef1e None admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:01.043271 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414d7b70-f2b6-44f2-9854-ca8cda69ef1e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:01.044898 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414d7b70-f2b6-44f2-9854-ca8cda69ef1e None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:01.053896 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f0705-1e0e-47a3-9bff-0237f5664a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:01.054616 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f0705-1e0e-47a3-9bff-0237f5664a3a None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:01.055258 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f0705-1e0e-47a3-9bff-0237f5664a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:01.055803 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f0705-1e0e-47a3-9bff-0237f5664a3a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:01.072704 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f0705-1e0e-47a3-9bff-0237f5664a3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:01.078920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-155f0705-1e0e-47a3-9bff-0237f5664a3a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:01.078920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:01.078920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:01.078920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:01.078920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:01.078920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:01.078920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:01.078920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:01.078920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:01.078920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:01.078920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:01.078920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:01.078920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:01.078920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:01.078920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:01.078920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:01.080319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:01.080319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:01.080319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:01.080319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:01.080319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:01.080319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:01.080319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:01.080319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:01.080319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:01.080319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:01.080319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:01.080319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:01.080319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:01.080319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:01.080319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:01.080319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:01.086987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:01.086987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:01.086987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:01.086987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:01.086987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:01.086987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:01.086987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:01.086987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:01.086987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:01.086987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:01.086987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:01.086987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:01.087696 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 55/108] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Sep 5 12:05:00 2022] POST /identity/v3/projects => generated 302 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:05:01.087776 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 54/109] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:01.087854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d7b70-f2b6-44f2-9854-ca8cda69ef1e None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:01.087927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956cf84-dc3f-4d83-99d2-0d97cda7e5f7 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:01.087927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956cf84-dc3f-4d83-99d2-0d97cda7e5f7 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:01.087927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956cf84-dc3f-4d83-99d2-0d97cda7e5f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:01.107366 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-9956cf84-dc3f-4d83-99d2-0d97cda7e5f7 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:01.109290 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9956cf84-dc3f-4d83-99d2-0d97cda7e5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:01.120988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:01.202292 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 55/110] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:01.203691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a049cb3-e1fb-451f-8f71-66e3dfef3c6d None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:01.203981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a049cb3-e1fb-451f-8f71-66e3dfef3c6d None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:01.204260 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a049cb3-e1fb-451f-8f71-66e3dfef3c6d None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:01.205926 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 56/111] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:01.206827 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebc974-f8d1-4457-b578-d1e729b41ca1 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:01.207108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebc974-f8d1-4457-b578-d1e729b41ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:01.207383 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebc974-f8d1-4457-b578-d1e729b41ca1 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:01.209008 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 57/112] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:01.209535 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ebc974-f8d1-4457-b578-d1e729b41ca1 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:01.264511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d7b70-f2b6-44f2-9854-ca8cda69ef1e None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:01.278223 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:01.322520 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d7001-f240-4afc-9118-f2abc4de504c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:01.323703 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d7001-f240-4afc-9118-f2abc4de504c None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:01.324196 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d7001-f240-4afc-9118-f2abc4de504c None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:01.324704 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d7001-f240-4afc-9118-f2abc4de504c None admin] PATH_INFO: `/v3/users/d39ae49df8b9460da658453f5e30dc67` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:01.354407 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0d7001-f240-4afc-9118-f2abc4de504c None admin] RBAC: Authorizing `identity:get_user(user_id=d39ae49df8b9460da658453f5e30dc67)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:01.357221 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0d7001-f240-4afc-9118-f2abc4de504c None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:01.361220 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 56/113] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:05:01 2022] GET /identity/v3/users/d39ae49df8b9460da658453f5e30dc67 => generated 289 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:01.363249 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f346e-8f3f-4c5a-a111-3e1a2249a973 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:01.363796 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f346e-8f3f-4c5a-a111-3e1a2249a973 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:01.364318 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f346e-8f3f-4c5a-a111-3e1a2249a973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:01.368657 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ebc974-f8d1-4457-b578-d1e729b41ca1 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:01.395139 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-ab4f346e-8f3f-4c5a-a111-3e1a2249a973 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:01.397140 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-ab4f346e-8f3f-4c5a-a111-3e1a2249a973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:01.410910 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af6085-9535-483f-ae51-e79e146a97ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:01.411553 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af6085-9535-483f-ae51-e79e146a97ca None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:01.411854 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af6085-9535-483f-ae51-e79e146a97ca None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:01.412140 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af6085-9535-483f-ae51-e79e146a97ca None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:01.412871 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af6085-9535-483f-ae51-e79e146a97ca None admin] RBAC: Adding query filter params (name=member) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:01.413184 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af6085-9535-483f-ae51-e79e146a97ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:01.415313 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af6085-9535-483f-ae51-e79e146a97ca None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:01.426344 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 58/114] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Sep 5 12:05:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:01.427937 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd380a-7a36-45e2-8dbd-dc1637c25375 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:01.428226 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd380a-7a36-45e2-8dbd-dc1637c25375 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:01.428508 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd380a-7a36-45e2-8dbd-dc1637c25375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:01.444632 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:01.453439 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-badd380a-7a36-45e2-8dbd-dc1637c25375 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:01.454845 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-badd380a-7a36-45e2-8dbd-dc1637c25375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:01.522770 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 57/115] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:01.531659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f346e-8f3f-4c5a-a111-3e1a2249a973 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:01.537343 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 59/116] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:01.538292 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd380a-7a36-45e2-8dbd-dc1637c25375 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:01.667935 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:01.672493 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f346e-8f3f-4c5a-a111-3e1a2249a973 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:01.731505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd380a-7a36-45e2-8dbd-dc1637c25375 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:01.755426 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211fd01d-f938-4fc6-89e8-3a522c860ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:01.756079 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211fd01d-f938-4fc6-89e8-3a522c860ab0 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:01.756365 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211fd01d-f938-4fc6-89e8-3a522c860ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:01.756644 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211fd01d-f938-4fc6-89e8-3a522c860ab0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:01.810059 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0e49a-f8b7-40aa-9e08-7965adb42f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:01.811180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0e49a-f8b7-40aa-9e08-7965adb42f4a None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:01.811927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0e49a-f8b7-40aa-9e08-7965adb42f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:01.812423 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0e49a-f8b7-40aa-9e08-7965adb42f4a None admin] PATH_INFO: `/v3/users/10881ece7d184527a88b151aac434c0c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:01.822491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0e49a-f8b7-40aa-9e08-7965adb42f4a None admin] RBAC: Authorizing `identity:get_user(user_id=10881ece7d184527a88b151aac434c0c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:01.824732 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0e49a-f8b7-40aa-9e08-7965adb42f4a None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:01.827103 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 60/117] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:05:01 2022] GET /identity/v3/users/10881ece7d184527a88b151aac434c0c => generated 275 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:01.828874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0e49a-f8b7-40aa-9e08-7965adb42f4a None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:01.841478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211fd01d-f938-4fc6-89e8-3a522c860ab0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:01.843606 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211fd01d-f938-4fc6-89e8-3a522c860ab0 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:01.851929 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:01.966680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:01.976651 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 58/118] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:05:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:05:01.978207 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211fd01d-f938-4fc6-89e8-3a522c860ab0 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:01.990914 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0e49a-f8b7-40aa-9e08-7965adb42f4a None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:02.030537 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7324c84-8305-4432-ac81-624c4c681db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:02.031719 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7324c84-8305-4432-ac81-624c4c681db9 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:02.032005 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7324c84-8305-4432-ac81-624c4c681db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:02.032284 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7324c84-8305-4432-ac81-624c4c681db9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:02.045757 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7324c84-8305-4432-ac81-624c4c681db9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:02.051219 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-b7324c84-8305-4432-ac81-624c4c681db9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:05:02.051219 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:02.051219 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:02.051219 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:02.051219 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:02.051219 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:02.051219 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:02.051219 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:02.051219 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:02.051219 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:02.051219 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:02.051219 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:02.051219 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:02.051219 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:02.051219 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:02.051219 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:02.055583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:02.055583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:02.055583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:02.055583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:02.055583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:02.055583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:02.055583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:02.055583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:02.055583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:02.055583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:02.055583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:02.055583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:02.055583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:02.055583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:02.055583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:02.055583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:02.056519 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:02.056519 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:02.056519 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:02.056519 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:02.056519 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:02.056519 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:02.056519 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:02.056519 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:02.056519 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:02.056519 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:02.056519 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:05:02.056519 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:02.056519 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 61/119] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:05:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:02.056519 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7324c84-8305-4432-ac81-624c4c681db9 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:02.128188 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211fd01d-f938-4fc6-89e8-3a522c860ab0 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:02.143984 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:02.171007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de6a2a1-a20a-4b28-8147-359b72b52cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:02.171716 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6a2a1-a20a-4b28-8147-359b72b52cab None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:02.172007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6a2a1-a20a-4b28-8147-359b72b52cab None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:02.172306 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6a2a1-a20a-4b28-8147-359b72b52cab None admin] PATH_INFO: `/v3/projects/405d2cd2fe68456ca034a57f24f6b015` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:02.173537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de6a2a1-a20a-4b28-8147-359b72b52cab None admin] RBAC: Authorizing `identity:get_project(project_id=405d2cd2fe68456ca034a57f24f6b015)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:02.175228 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de6a2a1-a20a-4b28-8147-359b72b52cab None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:02.176968 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 59/120] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:01 2022] GET /identity/v3/projects/405d2cd2fe68456ca034a57f24f6b015 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:02.177618 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de6a2a1-a20a-4b28-8147-359b72b52cab None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:02.194246 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7324c84-8305-4432-ac81-624c4c681db9 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:02.245982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dcbd91-3d3b-461d-8603-71d00f62dbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:02.247015 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcbd91-3d3b-461d-8603-71d00f62dbf6 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:02.248020 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcbd91-3d3b-461d-8603-71d00f62dbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:02.248307 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcbd91-3d3b-461d-8603-71d00f62dbf6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:02.259154 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:02.260787 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dcbd91-3d3b-461d-8603-71d00f62dbf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:02.267471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-30dcbd91-3d3b-461d-8603-71d00f62dbf6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:05:02.267471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:02.267471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:02.267471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:02.267471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:02.267471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:02.267471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:02.267471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:02.267471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:02.267471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:02.267471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:02.267471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:02.267471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:02.267471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:02.267471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:02.267471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:02.268426 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:02.268426 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:02.268426 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:02.268426 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:02.268426 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:02.268426 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:02.268426 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:02.268426 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:02.268426 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:02.268426 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:02.268426 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:02.268426 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:02.268426 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:02.268426 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:02.268426 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:02.268426 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:02.269201 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:02.269201 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:02.269201 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:02.269201 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:02.269201 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:02.269201 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:02.269201 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:02.269201 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:02.269201 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:02.269201 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:02.269201 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:05:02.269201 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:02.269201 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 62/121] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Sep 5 12:05:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:02.269860 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dcbd91-3d3b-461d-8603-71d00f62dbf6 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:02.307773 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de6a2a1-a20a-4b28-8147-359b72b52cab None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:02.352060 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18735ddc-bebf-4345-8aa6-f6fa1afd2ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:02.352753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18735ddc-bebf-4345-8aa6-f6fa1afd2ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:02.353040 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18735ddc-bebf-4345-8aa6-f6fa1afd2ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:02.353323 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18735ddc-bebf-4345-8aa6-f6fa1afd2ff9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:02.364334 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18735ddc-bebf-4345-8aa6-f6fa1afd2ff9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:02.370340 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-18735ddc-bebf-4345-8aa6-f6fa1afd2ff9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:02.370340 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:02.370340 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:02.370340 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:02.370340 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:02.370340 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:02.370340 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:02.370340 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:02.370340 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:02.370340 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:02.370340 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:02.370340 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:02.370340 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:02.370340 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:02.370340 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:02.370340 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:02.371339 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:02.371339 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:02.371339 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:02.371339 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:02.371339 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:02.371339 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:02.371339 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:02.371339 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:02.371339 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:02.371339 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:02.371339 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:02.371339 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:02.371339 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:02.371339 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:02.371339 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:02.371339 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:02.372150 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:02.372150 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:02.372150 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:02.372150 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:02.372150 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:02.372150 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:02.372150 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:02.372150 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:02.372150 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:02.372150 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:02.372150 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:02.372150 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:02.372150 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 60/122] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:02.372150 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18735ddc-bebf-4345-8aa6-f6fa1afd2ff9 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:02.401777 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dcbd91-3d3b-461d-8603-71d00f62dbf6 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:02.429860 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:02.443992 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662f01f-20b2-4b4d-8ad4-a812681a0e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:02.444632 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662f01f-20b2-4b4d-8ad4-a812681a0e30 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:02.444921 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662f01f-20b2-4b4d-8ad4-a812681a0e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:02.445200 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662f01f-20b2-4b4d-8ad4-a812681a0e30 None admin] PATH_INFO: `/v3/projects/405d2cd2fe68456ca034a57f24f6b015` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:02.446341 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662f01f-20b2-4b4d-8ad4-a812681a0e30 None admin] RBAC: Authorizing `identity:get_project(project_id=405d2cd2fe68456ca034a57f24f6b015)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:02.447856 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662f01f-20b2-4b4d-8ad4-a812681a0e30 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:02.451586 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 63/123] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:02 2022] GET /identity/v3/projects/405d2cd2fe68456ca034a57f24f6b015 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:02.451586 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662f01f-20b2-4b4d-8ad4-a812681a0e30 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:02.490743 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18735ddc-bebf-4345-8aa6-f6fa1afd2ff9 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:02.518467 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:02.533181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4838d8f4-b823-4a1f-9a5f-664a0b6b4b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:02.533809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838d8f4-b823-4a1f-9a5f-664a0b6b4b43 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:02.534093 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838d8f4-b823-4a1f-9a5f-664a0b6b4b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:02.534392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838d8f4-b823-4a1f-9a5f-664a0b6b4b43 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:02.535208 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4838d8f4-b823-4a1f-9a5f-664a0b6b4b43 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:02.535539 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4838d8f4-b823-4a1f-9a5f-664a0b6b4b43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:02.536964 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4838d8f4-b823-4a1f-9a5f-664a0b6b4b43 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:02.548630 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 61/124] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:02.549453 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4838d8f4-b823-4a1f-9a5f-664a0b6b4b43 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:02.561204 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662f01f-20b2-4b4d-8ad4-a812681a0e30 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:02.598651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8356d7b-8ee8-4114-b8f9-4c07c096ed8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:02.599355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8356d7b-8ee8-4114-b8f9-4c07c096ed8b None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:02.599672 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8356d7b-8ee8-4114-b8f9-4c07c096ed8b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:02.599950 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8356d7b-8ee8-4114-b8f9-4c07c096ed8b None admin] PATH_INFO: `/v3/projects/405d2cd2fe68456ca034a57f24f6b015/users/10881ece7d184527a88b151aac434c0c/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:02.601938 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8356d7b-8ee8-4114-b8f9-4c07c096ed8b None admin] RBAC: Authorizing `identity:create_grant(project_id=405d2cd2fe68456ca034a57f24f6b015, user_id=10881ece7d184527a88b151aac434c0c, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:02.603406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8356d7b-8ee8-4114-b8f9-4c07c096ed8b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:02.627484 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 64/125] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Sep 5 12:05:02 2022] PUT /identity/v3/projects/405d2cd2fe68456ca034a57f24f6b015/users/10881ece7d184527a88b151aac434c0c/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:05:02.627484 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8356d7b-8ee8-4114-b8f9-4c07c096ed8b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:02.690886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:02.699606 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4838d8f4-b823-4a1f-9a5f-664a0b6b4b43 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:02.738983 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10a1566-008a-45b9-aacd-50600de6b651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:02.739705 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a1566-008a-45b9-aacd-50600de6b651 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:02.739994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a1566-008a-45b9-aacd-50600de6b651 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:02.740331 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a1566-008a-45b9-aacd-50600de6b651 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:02.741044 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10a1566-008a-45b9-aacd-50600de6b651 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:02.741357 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10a1566-008a-45b9-aacd-50600de6b651 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:02.742850 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10a1566-008a-45b9-aacd-50600de6b651 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:02.753109 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 62/126] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:05:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:02.754067 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10a1566-008a-45b9-aacd-50600de6b651 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:02.770595 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8356d7b-8ee8-4114-b8f9-4c07c096ed8b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:02.818268 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:02.822333 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf329ea-fb6f-4f23-a3de-99539c8c8189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:02.823239 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf329ea-fb6f-4f23-a3de-99539c8c8189 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:02.823556 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf329ea-fb6f-4f23-a3de-99539c8c8189 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:02.823829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf329ea-fb6f-4f23-a3de-99539c8c8189 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:02.824542 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf329ea-fb6f-4f23-a3de-99539c8c8189 None admin] RBAC: Adding query filter params (name=member) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:02.824851 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf329ea-fb6f-4f23-a3de-99539c8c8189 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:02.826245 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf329ea-fb6f-4f23-a3de-99539c8c8189 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:02.837204 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 65/127] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Sep 5 12:05:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:02.837889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf329ea-fb6f-4f23-a3de-99539c8c8189 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:02.902517 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10a1566-008a-45b9-aacd-50600de6b651 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:02.941985 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bac9ca-0cfa-4139-b960-3d05c8fd400b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:02.942553 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bac9ca-0cfa-4139-b960-3d05c8fd400b None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:02.942842 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bac9ca-0cfa-4139-b960-3d05c8fd400b None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:02.943121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bac9ca-0cfa-4139-b960-3d05c8fd400b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:02.943895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bac9ca-0cfa-4139-b960-3d05c8fd400b None admin] RBAC: Adding query filter params (role.id=290929f17e764f019e358e7751365fd5, scope.project.id=405d2cd2fe68456ca034a57f24f6b015, user.id=d39ae49df8b9460da658453f5e30dc67) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:02.944306 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bac9ca-0cfa-4139-b960-3d05c8fd400b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:02.945902 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bac9ca-0cfa-4139-b960-3d05c8fd400b None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:02.952582 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 63/128] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:05:02 2022] GET /identity/v3/role_assignments?user.id=d39ae49df8b9460da658453f5e30dc67&scope.project.id=405d2cd2fe68456ca034a57f24f6b015&role.id=290929f17e764f019e358e7751365fd5 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:02.953244 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bac9ca-0cfa-4139-b960-3d05c8fd400b None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:02.960792 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf329ea-fb6f-4f23-a3de-99539c8c8189 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:02.994205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:03.058810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed16c2-7529-4f92-a95d-08bbdde8f4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:03.059959 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed16c2-7529-4f92-a95d-08bbdde8f4e5 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:03.060442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed16c2-7529-4f92-a95d-08bbdde8f4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:03.060930 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed16c2-7529-4f92-a95d-08bbdde8f4e5 None admin] PATH_INFO: `/v3/users/78575c7ceaa64d1b86e8eef0293f1e3c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:03.066745 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed16c2-7529-4f92-a95d-08bbdde8f4e5 None admin] RBAC: Authorizing `identity:get_user(user_id=78575c7ceaa64d1b86e8eef0293f1e3c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:03.072167 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed16c2-7529-4f92-a95d-08bbdde8f4e5 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:03.079875 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 66/129] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:05:02 2022] GET /identity/v3/users/78575c7ceaa64d1b86e8eef0293f1e3c => generated 247 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:03.082824 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed16c2-7529-4f92-a95d-08bbdde8f4e5 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:03.181274 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bac9ca-0cfa-4139-b960-3d05c8fd400b None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:03.196247 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:03.226565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff43af2-3c70-4014-b0f0-e4fbcfedd1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:03.227290 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff43af2-3c70-4014-b0f0-e4fbcfedd1ec None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:03.227611 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff43af2-3c70-4014-b0f0-e4fbcfedd1ec None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:03.227895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff43af2-3c70-4014-b0f0-e4fbcfedd1ec None admin] PATH_INFO: `/v3/users/10881ece7d184527a88b151aac434c0c` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:03.229125 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff43af2-3c70-4014-b0f0-e4fbcfedd1ec None admin] RBAC: Authorizing `identity:get_user(user_id=10881ece7d184527a88b151aac434c0c)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:03.230602 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff43af2-3c70-4014-b0f0-e4fbcfedd1ec None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:03.233190 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 64/130] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:05:02 2022] GET /identity/v3/users/10881ece7d184527a88b151aac434c0c => generated 275 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:03.236171 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff43af2-3c70-4014-b0f0-e4fbcfedd1ec None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:03.263452 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed16c2-7529-4f92-a95d-08bbdde8f4e5 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:03.302796 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33588959-ad05-4db8-adc8-3b94856901cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:03.303561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33588959-ad05-4db8-adc8-3b94856901cc None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:03.303856 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33588959-ad05-4db8-adc8-3b94856901cc None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:03.304133 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33588959-ad05-4db8-adc8-3b94856901cc None admin] PATH_INFO: `/v3/users/10881ece7d184527a88b151aac434c0c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:03.305279 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33588959-ad05-4db8-adc8-3b94856901cc None admin] RBAC: Authorizing `identity:get_user(user_id=10881ece7d184527a88b151aac434c0c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:03.306979 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33588959-ad05-4db8-adc8-3b94856901cc None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:03.308617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 67/131] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:05:03 2022] GET /identity/v3/users/10881ece7d184527a88b151aac434c0c => generated 275 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:03.310131 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16135744-73d5-497c-b53e-fe5f940b4e03 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:03.310424 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16135744-73d5-497c-b53e-fe5f940b4e03 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:03.310699 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16135744-73d5-497c-b53e-fe5f940b4e03 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:03.311986 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 68/132] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:03.316851 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16135744-73d5-497c-b53e-fe5f940b4e03 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:03.368926 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff43af2-3c70-4014-b0f0-e4fbcfedd1ec None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:03.377007 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:03.419915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6845532-c101-4bf2-b972-dc92d924ee4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:03.420623 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6845532-c101-4bf2-b972-dc92d924ee4a None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:03.421114 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6845532-c101-4bf2-b972-dc92d924ee4a None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:03.421404 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6845532-c101-4bf2-b972-dc92d924ee4a None admin] PATH_INFO: `/v3/projects/405d2cd2fe68456ca034a57f24f6b015` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:03.422869 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6845532-c101-4bf2-b972-dc92d924ee4a None admin] RBAC: Authorizing `identity:get_project(project_id=405d2cd2fe68456ca034a57f24f6b015)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:03.424377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6845532-c101-4bf2-b972-dc92d924ee4a None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:03.426518 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 65/133] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:03 2022] GET /identity/v3/projects/405d2cd2fe68456ca034a57f24f6b015 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:03.427721 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6845532-c101-4bf2-b972-dc92d924ee4a None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:03.441062 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16135744-73d5-497c-b53e-fe5f940b4e03 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:03.487287 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe161c6-b2fc-4102-a0de-de47d9c3fa41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:03.488566 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe161c6-b2fc-4102-a0de-de47d9c3fa41 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:03.489315 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe161c6-b2fc-4102-a0de-de47d9c3fa41 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:03.489890 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe161c6-b2fc-4102-a0de-de47d9c3fa41 None admin] PATH_INFO: `/v3/projects/405d2cd2fe68456ca034a57f24f6b015` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:03.491424 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe161c6-b2fc-4102-a0de-de47d9c3fa41 None admin] RBAC: Authorizing `identity:get_project(project_id=405d2cd2fe68456ca034a57f24f6b015)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:03.492753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:03.495913 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe161c6-b2fc-4102-a0de-de47d9c3fa41 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:03.499504 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 69/134] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:03 2022] GET /identity/v3/projects/405d2cd2fe68456ca034a57f24f6b015 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:03.501346 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389db76-472b-4f5e-80df-8e93b24663f8 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:03.501931 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389db76-472b-4f5e-80df-8e93b24663f8 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:03.502652 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389db76-472b-4f5e-80df-8e93b24663f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:03.523677 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-c389db76-472b-4f5e-80df-8e93b24663f8 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:03.525281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c389db76-472b-4f5e-80df-8e93b24663f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:03.563192 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6845532-c101-4bf2-b972-dc92d924ee4a None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:03.603076 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 70/135] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:03.604338 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389db76-472b-4f5e-80df-8e93b24663f8 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:03.606308 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe71555-d274-46cb-a289-0ed375080f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:03.606957 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe71555-d274-46cb-a289-0ed375080f0d None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:03.607245 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe71555-d274-46cb-a289-0ed375080f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:03.607547 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe71555-d274-46cb-a289-0ed375080f0d None admin] PATH_INFO: `/v3/projects/644903936a194df991e9309a715f6877` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:03.609914 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe71555-d274-46cb-a289-0ed375080f0d None admin] RBAC: Authorizing `identity:get_project(project_id=644903936a194df991e9309a715f6877)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:03.611402 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe71555-d274-46cb-a289-0ed375080f0d None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:03.613147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 66/136] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:03 2022] GET /identity/v3/projects/644903936a194df991e9309a715f6877 => generated 312 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:03.613915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe71555-d274-46cb-a289-0ed375080f0d None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:03.679064 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:03.751174 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389db76-472b-4f5e-80df-8e93b24663f8 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:03.789761 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe71555-d274-46cb-a289-0ed375080f0d None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:03.804445 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3a43a-16ca-4f07-ac3c-0e5fa9b6badb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:03.805501 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3a43a-16ca-4f07-ac3c-0e5fa9b6badb None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:03.806001 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3a43a-16ca-4f07-ac3c-0e5fa9b6badb None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:03.806489 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3a43a-16ca-4f07-ac3c-0e5fa9b6badb None admin] PATH_INFO: `/v3/projects/405d2cd2fe68456ca034a57f24f6b015/users/78575c7ceaa64d1b86e8eef0293f1e3c/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:03.812190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3a43a-16ca-4f07-ac3c-0e5fa9b6badb None admin] RBAC: Authorizing `identity:create_grant(project_id=405d2cd2fe68456ca034a57f24f6b015, user_id=78575c7ceaa64d1b86e8eef0293f1e3c, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:03.813921 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3a43a-16ca-4f07-ac3c-0e5fa9b6badb None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:03.829738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:03.837212 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 71/137] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Sep 5 12:05:03 2022] PUT /identity/v3/projects/405d2cd2fe68456ca034a57f24f6b015/users/78575c7ceaa64d1b86e8eef0293f1e3c/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:05:03.838191 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3a43a-16ca-4f07-ac3c-0e5fa9b6badb None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:03.866046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54368076-76a1-4d4d-8fb8-00adf3c262c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:03.866741 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54368076-76a1-4d4d-8fb8-00adf3c262c4 None admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:03.868495 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54368076-76a1-4d4d-8fb8-00adf3c262c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:03.868784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54368076-76a1-4d4d-8fb8-00adf3c262c4 None admin] PATH_INFO: `/v3/projects/405d2cd2fe68456ca034a57f24f6b015/users/10881ece7d184527a88b151aac434c0c/roles/20bf8c77837d43b590a2bd9a67d01f84` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:03.873597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54368076-76a1-4d4d-8fb8-00adf3c262c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=405d2cd2fe68456ca034a57f24f6b015, user_id=10881ece7d184527a88b151aac434c0c, role_id=20bf8c77837d43b590a2bd9a67d01f84)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:03.877330 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54368076-76a1-4d4d-8fb8-00adf3c262c4 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:03.904330 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 67/138] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Sep 5 12:05:03 2022] PUT /identity/v3/projects/405d2cd2fe68456ca034a57f24f6b015/users/10881ece7d184527a88b151aac434c0c/roles/20bf8c77837d43b590a2bd9a67d01f84 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:05:03.996525 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3a43a-16ca-4f07-ac3c-0e5fa9b6badb None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:04.008608 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:04.049223 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d87094a-5409-4add-bbba-e7ae3923c009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:04.049925 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87094a-5409-4add-bbba-e7ae3923c009 None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:04.050213 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87094a-5409-4add-bbba-e7ae3923c009 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:04.050558 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87094a-5409-4add-bbba-e7ae3923c009 None admin] PATH_INFO: `/v3/projects/644903936a194df991e9309a715f6877/users/10881ece7d184527a88b151aac434c0c/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:04.052755 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d87094a-5409-4add-bbba-e7ae3923c009 None admin] RBAC: Authorizing `identity:create_grant(project_id=644903936a194df991e9309a715f6877, user_id=10881ece7d184527a88b151aac434c0c, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:04.054248 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d87094a-5409-4add-bbba-e7ae3923c009 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:04.078581 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 72/139] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Sep 5 12:05:03 2022] PUT /identity/v3/projects/644903936a194df991e9309a715f6877/users/10881ece7d184527a88b151aac434c0c/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:05:04.250231 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54368076-76a1-4d4d-8fb8-00adf3c262c4 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:04.391037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54368076-76a1-4d4d-8fb8-00adf3c262c4 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:04.436760 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674e25a-b0fb-410e-9096-762a12430266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:04.437779 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674e25a-b0fb-410e-9096-762a12430266 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:04.438266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674e25a-b0fb-410e-9096-762a12430266 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:04.438742 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674e25a-b0fb-410e-9096-762a12430266 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:04.441102 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e674e25a-b0fb-410e-9096-762a12430266 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:04.442872 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e674e25a-b0fb-410e-9096-762a12430266 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:04.445192 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 68/140] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:05:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:04.453345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d87094a-5409-4add-bbba-e7ae3923c009 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:04.585276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d87094a-5409-4add-bbba-e7ae3923c009 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:04.622148 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd55dbd-a1ee-4de8-a593-5f7725ad8ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:04.622775 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd55dbd-a1ee-4de8-a593-5f7725ad8ef6 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:04.623166 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd55dbd-a1ee-4de8-a593-5f7725ad8ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:04.623502 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd55dbd-a1ee-4de8-a593-5f7725ad8ef6 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:04.624186 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd55dbd-a1ee-4de8-a593-5f7725ad8ef6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:04.625659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd55dbd-a1ee-4de8-a593-5f7725ad8ef6 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:04.659599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 73/141] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Sep 5 12:05:04 2022] POST /identity/v3/users => generated 283 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:05:05.318263 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b58dd8-d5ed-4a0b-a178-4d10cfe9d3d7 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:05.318775 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b58dd8-d5ed-4a0b-a178-4d10cfe9d3d7 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:05.318846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b58dd8-d5ed-4a0b-a178-4d10cfe9d3d7 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:05.320197 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 69/142] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:05.325952 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5e663-daf1-47da-974e-c2765e9a5e09 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:05.326242 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5e663-daf1-47da-974e-c2765e9a5e09 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:05.326522 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5e663-daf1-47da-974e-c2765e9a5e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:05.350420 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-81e5e663-daf1-47da-974e-c2765e9a5e09 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:05.351765 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-81e5e663-daf1-47da-974e-c2765e9a5e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:05.376938 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:05.448271 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 74/143] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:05.752448 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc224c-80d1-4e85-8c99-2e2de14546c6 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:05.753096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc224c-80d1-4e85-8c99-2e2de14546c6 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:05.753595 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc224c-80d1-4e85-8c99-2e2de14546c6 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:05.755165 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 70/144] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:05.760940 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12f67b-9d82-453f-9461-980b3706a2df None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:05.761447 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12f67b-9d82-453f-9461-980b3706a2df None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:05.761523 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12f67b-9d82-453f-9461-980b3706a2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:05.789037 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-8c12f67b-9d82-453f-9461-980b3706a2df None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:05.790271 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8c12f67b-9d82-453f-9461-980b3706a2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:05.819936 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfc224c-80d1-4e85-8c99-2e2de14546c6 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:05.912126 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:05.914577 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 75/145] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:06.024575 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfc224c-80d1-4e85-8c99-2e2de14546c6 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:06.073738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ecd41-f06e-4753-ba64-73f3cb07eeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:06.074391 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ecd41-f06e-4753-ba64-73f3cb07eeb5 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:06.074672 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ecd41-f06e-4753-ba64-73f3cb07eeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:06.074947 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ecd41-f06e-4753-ba64-73f3cb07eeb5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:06.098126 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ecd41-f06e-4753-ba64-73f3cb07eeb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:06.103769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-403ecd41-f06e-4753-ba64-73f3cb07eeb5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:05:06.103769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:06.103769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:06.103769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:06.103769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:06.103769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:06.103769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:06.103769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:06.103769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:06.103769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:06.103769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:06.103769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:06.103769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:06.103769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:06.103769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:06.103769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:06.104701 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:06.104701 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:06.104701 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:06.104701 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:06.104701 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:06.104701 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:06.104701 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:06.104701 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:06.104701 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:06.104701 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:06.104701 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:06.104701 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:06.104701 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:06.104701 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:06.104701 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:06.104701 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:06.105481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:06.105481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:06.105481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:06.105481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:06.105481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:06.105481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:06.105481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:06.105481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:06.105481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:06.105481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:06.105481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:05:06.105481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:06.105481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 71/146] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:05:06.111358 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12f67b-9d82-453f-9461-980b3706a2df None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:06.228942 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:06.247562 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12f67b-9d82-453f-9461-980b3706a2df None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:06.293468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d8152-fe7b-487e-b9dd-849ee040f359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:06.294092 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d8152-fe7b-487e-b9dd-849ee040f359 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:06.294419 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d8152-fe7b-487e-b9dd-849ee040f359 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:06.294695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d8152-fe7b-487e-b9dd-849ee040f359 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:06.295371 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d8152-fe7b-487e-b9dd-849ee040f359 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:06.295703 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d8152-fe7b-487e-b9dd-849ee040f359 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:06.297693 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d8152-fe7b-487e-b9dd-849ee040f359 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:06.308239 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 76/147] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Sep 5 12:05:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:06.314727 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ecd41-f06e-4753-ba64-73f3cb07eeb5 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:06.495016 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ecd41-f06e-4753-ba64-73f3cb07eeb5 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:06.499821 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:06.538329 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a0a4b6-cf7f-4e4d-9d32-da7c34dabd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:06.538994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a0a4b6-cf7f-4e4d-9d32-da7c34dabd0c None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:06.539313 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a0a4b6-cf7f-4e4d-9d32-da7c34dabd0c None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:06.539624 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a0a4b6-cf7f-4e4d-9d32-da7c34dabd0c None admin] PATH_INFO: `/v3/users/d39ae49df8b9460da658453f5e30dc67` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:06.540847 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a0a4b6-cf7f-4e4d-9d32-da7c34dabd0c None admin] RBAC: Authorizing `identity:get_user(user_id=d39ae49df8b9460da658453f5e30dc67)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:06.542343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a0a4b6-cf7f-4e4d-9d32-da7c34dabd0c None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:06.544198 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 72/148] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:05:06 2022] GET /identity/v3/users/d39ae49df8b9460da658453f5e30dc67 => generated 289 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:06.550419 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d8152-fe7b-487e-b9dd-849ee040f359 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:06.577915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a0a4b6-cf7f-4e4d-9d32-da7c34dabd0c None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:06.768889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a0a4b6-cf7f-4e4d-9d32-da7c34dabd0c None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:06.770769 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d8152-fe7b-487e-b9dd-849ee040f359 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:06.831122 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce7ec2-75ad-4ef6-88e5-4a1f6fc2cbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:06.831728 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce7ec2-75ad-4ef6-88e5-4a1f6fc2cbbb None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:06.832011 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce7ec2-75ad-4ef6-88e5-4a1f6fc2cbbb None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:06.832293 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce7ec2-75ad-4ef6-88e5-4a1f6fc2cbbb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:06.840565 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d91d6c-a83b-4b42-aee6-3b3a1bfccafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:06.841192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d91d6c-a83b-4b42-aee6-3b3a1bfccafd None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:06.841472 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d91d6c-a83b-4b42-aee6-3b3a1bfccafd None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:06.841917 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d91d6c-a83b-4b42-aee6-3b3a1bfccafd None admin] PATH_INFO: `/v3/projects/405d2cd2fe68456ca034a57f24f6b015` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:06.843899 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d91d6c-a83b-4b42-aee6-3b3a1bfccafd None admin] RBAC: Authorizing `identity:get_project(project_id=405d2cd2fe68456ca034a57f24f6b015)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:06.845549 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d91d6c-a83b-4b42-aee6-3b3a1bfccafd None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:06.847403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 77/149] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:06 2022] GET /identity/v3/projects/405d2cd2fe68456ca034a57f24f6b015 => generated 298 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:06.856832 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cce7ec2-75ad-4ef6-88e5-4a1f6fc2cbbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:06.866864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-8cce7ec2-75ad-4ef6-88e5-4a1f6fc2cbbb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:06.866864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:06.866864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:06.866864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:06.866864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:06.866864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:06.866864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:06.866864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:06.866864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:06.866864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:06.866864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:06.866864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:06.866864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:06.866864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:06.866864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:06.866864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:06.868045 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:06.868045 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:06.868045 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:06.868045 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:06.868045 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:06.868045 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:06.868045 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:06.868045 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:06.868045 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:06.868045 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:06.868045 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:06.868045 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:06.868045 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:06.868045 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:06.868045 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:06.868045 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:06.869209 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:06.869209 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:06.869209 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:06.869209 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:06.869209 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:06.869209 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:06.869209 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:06.869209 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:06.869209 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:06.869209 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:06.869209 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:06.869209 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:06.869209 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 73/150] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:05:06.873803 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d91d6c-a83b-4b42-aee6-3b3a1bfccafd None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:06.874533 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce7ec2-75ad-4ef6-88e5-4a1f6fc2cbbb None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:06.927930 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:06.939957 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:07.041343 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d91d6c-a83b-4b42-aee6-3b3a1bfccafd None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:07.087893 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce7ec2-75ad-4ef6-88e5-4a1f6fc2cbbb None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:07.100275 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28480c58-74d9-42e5-9027-9b1a69a2cc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:07.103635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28480c58-74d9-42e5-9027-9b1a69a2cc8f None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:07.103635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28480c58-74d9-42e5-9027-9b1a69a2cc8f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:07.103635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28480c58-74d9-42e5-9027-9b1a69a2cc8f None admin] PATH_INFO: `/v3/projects/405d2cd2fe68456ca034a57f24f6b015/users/d39ae49df8b9460da658453f5e30dc67/roles/290929f17e764f019e358e7751365fd5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:07.108299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28480c58-74d9-42e5-9027-9b1a69a2cc8f None admin] RBAC: Authorizing `identity:create_grant(project_id=405d2cd2fe68456ca034a57f24f6b015, user_id=d39ae49df8b9460da658453f5e30dc67, role_id=290929f17e764f019e358e7751365fd5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:07.109809 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28480c58-74d9-42e5-9027-9b1a69a2cc8f None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:07.137863 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 78/151] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Sep 5 12:05:06 2022] PUT /identity/v3/projects/405d2cd2fe68456ca034a57f24f6b015/users/d39ae49df8b9460da658453f5e30dc67/roles/290929f17e764f019e358e7751365fd5 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:05:07.146524 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5f21a-334a-4801-88ac-d65cbba67140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:07.147255 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5f21a-334a-4801-88ac-d65cbba67140 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:07.147585 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5f21a-334a-4801-88ac-d65cbba67140 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:07.147945 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5f21a-334a-4801-88ac-d65cbba67140 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:07.148644 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c5f21a-334a-4801-88ac-d65cbba67140 None admin] RBAC: Adding query filter params (name=member) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:07.149125 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c5f21a-334a-4801-88ac-d65cbba67140 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:07.150564 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c5f21a-334a-4801-88ac-d65cbba67140 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:07.166317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 74/152] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Sep 5 12:05:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:07.172404 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28480c58-74d9-42e5-9027-9b1a69a2cc8f None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:07.282406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:07.313301 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28480c58-74d9-42e5-9027-9b1a69a2cc8f None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:07.354749 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ba22c-3fe1-4fb6-8430-2e9db48fef8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:07.355324 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ba22c-3fe1-4fb6-8430-2e9db48fef8e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:07.355628 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ba22c-3fe1-4fb6-8430-2e9db48fef8e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:07.355906 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ba22c-3fe1-4fb6-8430-2e9db48fef8e None admin] PATH_INFO: `/v3/users/10881ece7d184527a88b151aac434c0c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:07.357067 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ba22c-3fe1-4fb6-8430-2e9db48fef8e None admin] RBAC: Authorizing `identity:get_user(user_id=10881ece7d184527a88b151aac434c0c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:07.358561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ba22c-3fe1-4fb6-8430-2e9db48fef8e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:07.360233 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 79/153] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:05:07 2022] GET /identity/v3/users/10881ece7d184527a88b151aac434c0c => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:07.366128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5f21a-334a-4801-88ac-d65cbba67140 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:07.497993 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:07.506469 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5f21a-334a-4801-88ac-d65cbba67140 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:07.513645 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f2088-9629-4a35-b246-cce461b2f7c3 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:07.514239 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f2088-9629-4a35-b246-cce461b2f7c3 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:07.515275 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f2088-9629-4a35-b246-cce461b2f7c3 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:07.519639 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 80/154] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:07.529195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9c0ad-560c-4054-947d-92f7d049773f None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:07.529785 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9c0ad-560c-4054-947d-92f7d049773f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:07.530308 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9c0ad-560c-4054-947d-92f7d049773f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:07.546518 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e112b15-8586-49db-9308-2af1687acdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:07.547136 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e112b15-8586-49db-9308-2af1687acdcb None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:07.547414 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e112b15-8586-49db-9308-2af1687acdcb None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:07.547723 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e112b15-8586-49db-9308-2af1687acdcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:07.555036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-c7c9c0ad-560c-4054-947d-92f7d049773f None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:07.556539 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c7c9c0ad-560c-4054-947d-92f7d049773f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:07.629956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e112b15-8586-49db-9308-2af1687acdcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:07.631829 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e112b15-8586-49db-9308-2af1687acdcb None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:07.670349 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 81/155] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:07.673470 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1c03d-2b9e-4c49-9b95-dfd9fb2606d8 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:07.673963 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1c03d-2b9e-4c49-9b95-dfd9fb2606d8 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:07.674434 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1c03d-2b9e-4c49-9b95-dfd9fb2606d8 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:07.675986 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 82/156] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:07.681505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45face39-bbad-43c9-b0e9-2ef4bcdaad24 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:07.682214 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45face39-bbad-43c9-b0e9-2ef4bcdaad24 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:07.682704 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45face39-bbad-43c9-b0e9-2ef4bcdaad24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:07.706164 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-45face39-bbad-43c9-b0e9-2ef4bcdaad24 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:07.707738 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-45face39-bbad-43c9-b0e9-2ef4bcdaad24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:07.736867 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:07.747023 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 75/157] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:05:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:05:07.753623 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e112b15-8586-49db-9308-2af1687acdcb None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:07.795755 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 83/158] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:05:07.866745 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e112b15-8586-49db-9308-2af1687acdcb None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:07.904269 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8175444-eb9e-4043-bbdb-39a74c7a43b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:07.904821 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8175444-eb9e-4043-bbdb-39a74c7a43b8 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:07.905097 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8175444-eb9e-4043-bbdb-39a74c7a43b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:07.905454 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8175444-eb9e-4043-bbdb-39a74c7a43b8 None admin] PATH_INFO: `/v3/projects/405d2cd2fe68456ca034a57f24f6b015` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:07.906600 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8175444-eb9e-4043-bbdb-39a74c7a43b8 None admin] RBAC: Authorizing `identity:get_project(project_id=405d2cd2fe68456ca034a57f24f6b015)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:07.908155 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8175444-eb9e-4043-bbdb-39a74c7a43b8 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:07.909808 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 76/159] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:07 2022] GET /identity/v3/projects/405d2cd2fe68456ca034a57f24f6b015 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:07.915699 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45face39-bbad-43c9-b0e9-2ef4bcdaad24 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:07.918319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:08.029437 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45face39-bbad-43c9-b0e9-2ef4bcdaad24 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:08.080427 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed43cb9-1088-4fda-bcda-5186be9d957d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:08.081058 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed43cb9-1088-4fda-bcda-5186be9d957d None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:08.081336 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed43cb9-1088-4fda-bcda-5186be9d957d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:08.081610 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed43cb9-1088-4fda-bcda-5186be9d957d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:08.082379 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed43cb9-1088-4fda-bcda-5186be9d957d None admin] RBAC: Adding query filter params (role.id=ede2986592354e9888dde3123b2d8c82, scope.project.id=405d2cd2fe68456ca034a57f24f6b015, user.id=10881ece7d184527a88b151aac434c0c) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:08.082707 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed43cb9-1088-4fda-bcda-5186be9d957d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:08.084335 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed43cb9-1088-4fda-bcda-5186be9d957d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:08.091134 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 84/160] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:05:07 2022] GET /identity/v3/role_assignments?user.id=10881ece7d184527a88b151aac434c0c&scope.project.id=405d2cd2fe68456ca034a57f24f6b015&role.id=ede2986592354e9888dde3123b2d8c82 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:08.104000 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1abaf2b-53d2-4ef7-8159-ee8b90ab8aac None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:08.104300 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1abaf2b-53d2-4ef7-8159-ee8b90ab8aac None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:08.104568 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1abaf2b-53d2-4ef7-8159-ee8b90ab8aac None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:08.106002 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 77/161] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:08.125651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2cbb1-4a2f-4046-aa78-e4c7b698cc08 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:08.126012 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2cbb1-4a2f-4046-aa78-e4c7b698cc08 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:08.126201 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2cbb1-4a2f-4046-aa78-e4c7b698cc08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:08.145988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-ecb2cbb1-4a2f-4046-aa78-e4c7b698cc08 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:08.147187 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ecb2cbb1-4a2f-4046-aa78-e4c7b698cc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:08.216870 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:08.227290 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 85/162] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:08.332371 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1abaf2b-53d2-4ef7-8159-ee8b90ab8aac None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:08.362003 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2cbb1-4a2f-4046-aa78-e4c7b698cc08 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:08.440939 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:08.515236 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1abaf2b-53d2-4ef7-8159-ee8b90ab8aac None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:08.551731 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119b750-5a70-4c28-b1ba-c5e0fb1d0be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:08.552377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119b750-5a70-4c28-b1ba-c5e0fb1d0be9 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:08.552657 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119b750-5a70-4c28-b1ba-c5e0fb1d0be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:08.552932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119b750-5a70-4c28-b1ba-c5e0fb1d0be9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:08.573560 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2119b750-5a70-4c28-b1ba-c5e0fb1d0be9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:08.584166 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-2119b750-5a70-4c28-b1ba-c5e0fb1d0be9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:05:08.584166 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:08.584166 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:08.584166 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:08.584166 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:08.584166 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:08.584166 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:08.584166 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:08.584166 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:08.584166 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:08.584166 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:08.584166 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:08.584166 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:08.584166 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:08.584166 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:08.584166 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:08.585085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:08.585085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:08.585085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:08.585085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:08.585085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:08.585085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:08.585085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:08.585085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:08.585085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:08.585085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:08.585085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:08.585085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:08.585085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:08.585085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:08.585085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:08.585085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:08.586532 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:08.586532 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:08.586532 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:08.586532 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:08.586532 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:08.586532 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:08.586532 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:08.586532 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:08.586532 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:08.586532 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:08.586532 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:05:08.586532 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:08.586532 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 78/163] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Sep 5 12:05:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:08.593011 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119b750-5a70-4c28-b1ba-c5e0fb1d0be9 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:08.600584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2cbb1-4a2f-4046-aa78-e4c7b698cc08 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:08.675383 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf44cc8-88ea-4173-992d-b2d92ae200a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:08.676060 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf44cc8-88ea-4173-992d-b2d92ae200a3 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:08.676338 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf44cc8-88ea-4173-992d-b2d92ae200a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:08.676612 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf44cc8-88ea-4173-992d-b2d92ae200a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:08.696768 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf44cc8-88ea-4173-992d-b2d92ae200a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:08.702633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-2bf44cc8-88ea-4173-992d-b2d92ae200a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:05:08.702633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:08.702633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:08.702633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:08.702633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:08.702633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:08.702633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:08.702633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:08.702633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:08.702633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:08.702633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:08.702633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:08.702633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:08.702633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:08.702633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:08.702633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:08.703651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:08.703651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:08.703651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:08.703651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:08.703651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:08.703651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:08.703651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:08.703651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:08.703651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:08.703651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:08.703651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:08.703651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:08.703651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:08.703651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:08.703651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:08.703651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:08.704454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:08.704454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:08.704454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:08.704454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:08.704454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:08.704454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:08.704454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:08.704454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:08.704454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:08.704454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:08.704454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:05:08.704454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:08.706904 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 86/164] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:05:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:08.713624 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf44cc8-88ea-4173-992d-b2d92ae200a3 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:08.750215 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119b750-5a70-4c28-b1ba-c5e0fb1d0be9 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:08.775582 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:08.789716 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e984fe-21de-4d63-9759-55d4f555f483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:08.790356 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e984fe-21de-4d63-9759-55d4f555f483 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:08.790883 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e984fe-21de-4d63-9759-55d4f555f483 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:08.791317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e984fe-21de-4d63-9759-55d4f555f483 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:08.792217 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e984fe-21de-4d63-9759-55d4f555f483 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:08.792682 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e984fe-21de-4d63-9759-55d4f555f483 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:08.794279 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e984fe-21de-4d63-9759-55d4f555f483 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:08.804425 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 79/165] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:05:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:08.805651 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e984fe-21de-4d63-9759-55d4f555f483 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:08.818064 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:08.896674 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf44cc8-88ea-4173-992d-b2d92ae200a3 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:08.913605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e984fe-21de-4d63-9759-55d4f555f483 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:08.949124 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421f3c5-2523-4b5e-9194-011d21e48c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:08.950030 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421f3c5-2523-4b5e-9194-011d21e48c71 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:08.950522 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421f3c5-2523-4b5e-9194-011d21e48c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:08.950975 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421f3c5-2523-4b5e-9194-011d21e48c71 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:08.951861 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421f3c5-2523-4b5e-9194-011d21e48c71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:08.952451 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421f3c5-2523-4b5e-9194-011d21e48c71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:08.954126 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421f3c5-2523-4b5e-9194-011d21e48c71 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:08.958003 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a7f51-1071-4d82-afdd-0f67f1acf633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:08.958720 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a7f51-1071-4d82-afdd-0f67f1acf633 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:08.959071 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a7f51-1071-4d82-afdd-0f67f1acf633 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:08.959413 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a7f51-1071-4d82-afdd-0f67f1acf633 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:08.964671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 80/166] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:08.965820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421f3c5-2523-4b5e-9194-011d21e48c71 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:08.985452 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a7f51-1071-4d82-afdd-0f67f1acf633 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:08.990836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-856a7f51-1071-4d82-afdd-0f67f1acf633 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:08.990836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:08.990836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:08.990836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:08.990836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:08.990836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:08.990836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:08.990836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:08.990836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:08.990836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:08.990836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:08.990836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:08.990836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:08.990836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:08.990836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:08.990836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:08.991810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:08.991810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:08.991810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:08.991810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:08.991810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:08.991810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:08.991810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:08.991810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:08.991810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:08.991810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:08.991810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:08.991810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:08.991810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:08.991810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:08.991810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:08.991810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:08.992681 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:08.992681 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:08.992681 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:08.992681 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:08.992681 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:08.992681 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:08.992681 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:08.992681 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:08.992681 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:08.992681 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:08.992681 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:08.992681 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:08.992681 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 87/167] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:08.993349 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a7f51-1071-4d82-afdd-0f67f1acf633 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:09.034295 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:09.090861 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421f3c5-2523-4b5e-9194-011d21e48c71 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:09.129504 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269fbffc-ce4c-4a2c-9446-270b8dd3621f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:09.130445 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269fbffc-ce4c-4a2c-9446-270b8dd3621f None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:09.130935 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269fbffc-ce4c-4a2c-9446-270b8dd3621f None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:09.131389 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269fbffc-ce4c-4a2c-9446-270b8dd3621f None admin] PATH_INFO: `/v3/users/10881ece7d184527a88b151aac434c0c` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:09.132762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269fbffc-ce4c-4a2c-9446-270b8dd3621f None admin] RBAC: Authorizing `identity:get_user(user_id=10881ece7d184527a88b151aac434c0c)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:09.134419 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269fbffc-ce4c-4a2c-9446-270b8dd3621f None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:09.136275 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 81/168] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:05:08 2022] GET /identity/v3/users/10881ece7d184527a88b151aac434c0c => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:09.137459 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269fbffc-ce4c-4a2c-9446-270b8dd3621f None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:09.194287 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a7f51-1071-4d82-afdd-0f67f1acf633 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:09.231373 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:09.243472 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269fbffc-ce4c-4a2c-9446-270b8dd3621f None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:09.260081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2c470-4d3f-418e-bbf5-62889353120b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:09.261030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2c470-4d3f-418e-bbf5-62889353120b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:09.261514 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2c470-4d3f-418e-bbf5-62889353120b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:09.261959 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2c470-4d3f-418e-bbf5-62889353120b None admin] PATH_INFO: `/v3/users/78575c7ceaa64d1b86e8eef0293f1e3c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:09.265098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f2c470-4d3f-418e-bbf5-62889353120b None admin] RBAC: Authorizing `identity:get_user(user_id=78575c7ceaa64d1b86e8eef0293f1e3c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:09.266736 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f2c470-4d3f-418e-bbf5-62889353120b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:09.268564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 88/169] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:05:08 2022] GET /identity/v3/users/78575c7ceaa64d1b86e8eef0293f1e3c => generated 247 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:09.269528 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2c470-4d3f-418e-bbf5-62889353120b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:09.276232 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:09.310001 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2e72a-d718-40ab-8069-1e58b192ca23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:09.311620 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2e72a-d718-40ab-8069-1e58b192ca23 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:09.312488 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2e72a-d718-40ab-8069-1e58b192ca23 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:09.313241 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2e72a-d718-40ab-8069-1e58b192ca23 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:09.314591 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2e72a-d718-40ab-8069-1e58b192ca23 None admin] RBAC: Adding query filter params (name=member) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:09.315085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2e72a-d718-40ab-8069-1e58b192ca23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:09.320998 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2e72a-d718-40ab-8069-1e58b192ca23 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:09.337507 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 82/170] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Sep 5 12:05:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:09.338789 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2e72a-d718-40ab-8069-1e58b192ca23 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:09.430399 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2c470-4d3f-418e-bbf5-62889353120b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:09.474694 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21bd1a-88e5-4dcf-8ff6-4f43b49fffd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:09.475648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21bd1a-88e5-4dcf-8ff6-4f43b49fffd3 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:09.478159 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21bd1a-88e5-4dcf-8ff6-4f43b49fffd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:09.478654 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21bd1a-88e5-4dcf-8ff6-4f43b49fffd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:09.553664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2e72a-d718-40ab-8069-1e58b192ca23 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:09.583241 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:09.610757 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21bd1a-88e5-4dcf-8ff6-4f43b49fffd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:09.612799 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21bd1a-88e5-4dcf-8ff6-4f43b49fffd3 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:09.625582 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab0cb66-0073-4ef8-8655-b0b9b5d01523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:09.626218 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0cb66-0073-4ef8-8655-b0b9b5d01523 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:09.626497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0cb66-0073-4ef8-8655-b0b9b5d01523 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:09.626766 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0cb66-0073-4ef8-8655-b0b9b5d01523 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:09.712061 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:09.724522 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 89/171] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:05:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:05:09.725702 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21bd1a-88e5-4dcf-8ff6-4f43b49fffd3 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:09.751303 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab0cb66-0073-4ef8-8655-b0b9b5d01523 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:09.761155 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab0cb66-0073-4ef8-8655-b0b9b5d01523 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:09.819397 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:09.836440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21bd1a-88e5-4dcf-8ff6-4f43b49fffd3 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:09.877057 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc89a6d-748e-435b-9347-a013d3bc4ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:09.878060 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc89a6d-748e-435b-9347-a013d3bc4ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:09.878544 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc89a6d-748e-435b-9347-a013d3bc4ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:09.879016 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc89a6d-748e-435b-9347-a013d3bc4ef8 None admin] PATH_INFO: `/v3/users/10881ece7d184527a88b151aac434c0c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:09.880519 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc89a6d-748e-435b-9347-a013d3bc4ef8 None admin] RBAC: Authorizing `identity:get_user(user_id=10881ece7d184527a88b151aac434c0c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:09.882172 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc89a6d-748e-435b-9347-a013d3bc4ef8 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:09.884021 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 90/172] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:05:09 2022] GET /identity/v3/users/10881ece7d184527a88b151aac434c0c => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:09.885020 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc89a6d-748e-435b-9347-a013d3bc4ef8 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:09.995682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc89a6d-748e-435b-9347-a013d3bc4ef8 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:10.037518 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 83/173] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:05:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 699 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:05:10.038775 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d429cb-ecc2-4689-9ad9-74b30e4c32e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:10.039630 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c802d26-04e8-430f-b348-d9f1f2fe861e None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:10.039873 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d429cb-ecc2-4689-9ad9-74b30e4c32e8 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:10.039994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c802d26-04e8-430f-b348-d9f1f2fe861e None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:10.040318 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c802d26-04e8-430f-b348-d9f1f2fe861e None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:10.040691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d429cb-ecc2-4689-9ad9-74b30e4c32e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:10.041202 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d429cb-ecc2-4689-9ad9-74b30e4c32e8 None admin] PATH_INFO: `/v3/projects/405d2cd2fe68456ca034a57f24f6b015` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:10.042887 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d429cb-ecc2-4689-9ad9-74b30e4c32e8 None admin] RBAC: Authorizing `identity:get_project(project_id=405d2cd2fe68456ca034a57f24f6b015)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:10.043226 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 84/174] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:10.044098 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c802d26-04e8-430f-b348-d9f1f2fe861e None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:10.045438 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d429cb-ecc2-4689-9ad9-74b30e4c32e8 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:10.048128 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 91/175] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:09 2022] GET /identity/v3/projects/405d2cd2fe68456ca034a57f24f6b015 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:10.050491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772394c-2eec-454d-a1fd-c69e56e270e7 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:10.050995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772394c-2eec-454d-a1fd-c69e56e270e7 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:10.051505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772394c-2eec-454d-a1fd-c69e56e270e7 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:10.053206 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 92/176] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:10.054105 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772394c-2eec-454d-a1fd-c69e56e270e7 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:10.082925 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:10.161297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:10.189772 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772394c-2eec-454d-a1fd-c69e56e270e7 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:10.196151 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c802d26-04e8-430f-b348-d9f1f2fe861e None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:10.235341 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69085476-9fbd-4fa7-b461-207bfb1072cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:10.236007 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69085476-9fbd-4fa7-b461-207bfb1072cc None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:10.236284 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69085476-9fbd-4fa7-b461-207bfb1072cc None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:10.236554 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69085476-9fbd-4fa7-b461-207bfb1072cc None admin] PATH_INFO: `/v3/projects/405d2cd2fe68456ca034a57f24f6b015` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:10.238268 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69085476-9fbd-4fa7-b461-207bfb1072cc None admin] RBAC: Authorizing `identity:get_project(project_id=405d2cd2fe68456ca034a57f24f6b015)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:10.239454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69085476-9fbd-4fa7-b461-207bfb1072cc None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:10.245920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 93/177] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:10 2022] GET /identity/v3/projects/405d2cd2fe68456ca034a57f24f6b015 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:10.245920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0885d-335c-41f1-9a40-0574666ff77f None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:10.245920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0885d-335c-41f1-9a40-0574666ff77f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:10.245920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0885d-335c-41f1-9a40-0574666ff77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:10.248575 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f864da-5aa5-453c-86bf-bb30bec8f109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:10.249766 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f864da-5aa5-453c-86bf-bb30bec8f109 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:10.250043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f864da-5aa5-453c-86bf-bb30bec8f109 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:10.250314 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f864da-5aa5-453c-86bf-bb30bec8f109 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:10.266620 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-faa0885d-335c-41f1-9a40-0574666ff77f None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:10.268723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-faa0885d-335c-41f1-9a40-0574666ff77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:10.323312 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f864da-5aa5-453c-86bf-bb30bec8f109 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:10.324961 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f864da-5aa5-453c-86bf-bb30bec8f109 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:10.342043 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 94/178] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:10.347194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0885d-335c-41f1-9a40-0574666ff77f None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:10.378142 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:10.415969 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:10.436456 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 85/179] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:05:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:05:10.438250 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea953cb-f6d9-4e84-9816-eb2005cd3f21 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:10.438699 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea953cb-f6d9-4e84-9816-eb2005cd3f21 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:10.439155 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea953cb-f6d9-4e84-9816-eb2005cd3f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:10.457653 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-4ea953cb-f6d9-4e84-9816-eb2005cd3f21 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:10.459081 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-4ea953cb-f6d9-4e84-9816-eb2005cd3f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:10.490300 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0885d-335c-41f1-9a40-0574666ff77f None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:10.531189 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3d0ce7-b3e9-4bd5-af4f-da0b2f65e756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:10.532227 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3d0ce7-b3e9-4bd5-af4f-da0b2f65e756 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:10.532778 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3d0ce7-b3e9-4bd5-af4f-da0b2f65e756 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:10.533253 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3d0ce7-b3e9-4bd5-af4f-da0b2f65e756 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:10.534185 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3d0ce7-b3e9-4bd5-af4f-da0b2f65e756 None admin] RBAC: Adding query filter params (role.id=20bf8c77837d43b590a2bd9a67d01f84, scope.project.id=405d2cd2fe68456ca034a57f24f6b015, user.id=10881ece7d184527a88b151aac434c0c) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:10.534782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3d0ce7-b3e9-4bd5-af4f-da0b2f65e756 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:10.536529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3d0ce7-b3e9-4bd5-af4f-da0b2f65e756 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:10.538939 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 86/180] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:10.540978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed763b-ea76-4bd3-b66c-b4e645debe56 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:10.541430 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed763b-ea76-4bd3-b66c-b4e645debe56 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:10.542148 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed763b-ea76-4bd3-b66c-b4e645debe56 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:10.543862 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 87/181] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:10.544715 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed763b-ea76-4bd3-b66c-b4e645debe56 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:10.552198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 95/182] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:05:10 2022] GET /identity/v3/role_assignments?user.id=10881ece7d184527a88b151aac434c0c&scope.project.id=405d2cd2fe68456ca034a57f24f6b015&role.id=20bf8c77837d43b590a2bd9a67d01f84 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:10.553345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3d0ce7-b3e9-4bd5-af4f-da0b2f65e756 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:10.674766 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed763b-ea76-4bd3-b66c-b4e645debe56 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:10.682998 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:10.699778 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:10.714566 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1afc54-43ed-4c96-811a-ab59abbc1884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:10.714846 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3d0ce7-b3e9-4bd5-af4f-da0b2f65e756 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:10.715330 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1afc54-43ed-4c96-811a-ab59abbc1884 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:10.715731 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1afc54-43ed-4c96-811a-ab59abbc1884 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:10.716070 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1afc54-43ed-4c96-811a-ab59abbc1884 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:10.717197 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1afc54-43ed-4c96-811a-ab59abbc1884 None admin] RBAC: Adding query filter params (role.id=618c48706f7f445abeddf7eb2f8e927a, scope.project.id=405d2cd2fe68456ca034a57f24f6b015, user.id=78575c7ceaa64d1b86e8eef0293f1e3c) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:10.717575 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1afc54-43ed-4c96-811a-ab59abbc1884 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:10.719797 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1afc54-43ed-4c96-811a-ab59abbc1884 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:10.726891 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 88/183] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:05:10 2022] GET /identity/v3/role_assignments?user.id=78575c7ceaa64d1b86e8eef0293f1e3c&scope.project.id=405d2cd2fe68456ca034a57f24f6b015&role.id=618c48706f7f445abeddf7eb2f8e927a => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:10.728303 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c22aa6-4e57-42b3-b628-bc1ac9879506 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:10.728762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c22aa6-4e57-42b3-b628-bc1ac9879506 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:10.729106 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c22aa6-4e57-42b3-b628-bc1ac9879506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:10.749132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-a3c22aa6-4e57-42b3-b628-bc1ac9879506 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:10.750465 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-a3c22aa6-4e57-42b3-b628-bc1ac9879506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:10.772732 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e240a60-ce10-48c3-bfe7-7bddc4300227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:10.773489 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e240a60-ce10-48c3-bfe7-7bddc4300227 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:10.773776 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e240a60-ce10-48c3-bfe7-7bddc4300227 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:10.774108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e240a60-ce10-48c3-bfe7-7bddc4300227 None admin] PATH_INFO: `/v3/projects/644903936a194df991e9309a715f6877` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:10.775335 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e240a60-ce10-48c3-bfe7-7bddc4300227 None admin] RBAC: Authorizing `identity:get_project(project_id=644903936a194df991e9309a715f6877)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:10.777094 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e240a60-ce10-48c3-bfe7-7bddc4300227 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:10.779553 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 96/184] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:10 2022] GET /identity/v3/projects/644903936a194df991e9309a715f6877 => generated 312 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:10.780845 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e240a60-ce10-48c3-bfe7-7bddc4300227 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:10.830383 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 89/185] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:10.837017 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604fe11-d255-41f9-a2bf-169f76484c37 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:10.837562 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604fe11-d255-41f9-a2bf-169f76484c37 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:10.838009 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604fe11-d255-41f9-a2bf-169f76484c37 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:10.839601 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 90/186] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:10 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:10.840704 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604fe11-d255-41f9-a2bf-169f76484c37 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:10.904897 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e240a60-ce10-48c3-bfe7-7bddc4300227 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:10.950437 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f434e3b-a6a3-48e3-92f7-184da5eb22e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:10.950992 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f434e3b-a6a3-48e3-92f7-184da5eb22e2 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:10.951433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f434e3b-a6a3-48e3-92f7-184da5eb22e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:10.951910 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f434e3b-a6a3-48e3-92f7-184da5eb22e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:10.964579 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f434e3b-a6a3-48e3-92f7-184da5eb22e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:10.970820 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-4f434e3b-a6a3-48e3-92f7-184da5eb22e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:05:10.970820 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:10.970820 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:10.970820 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:10.970820 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:10.970820 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:10.970820 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:10.970820 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:10.970820 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:10.970820 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:10.970820 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:10.970820 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:10.970820 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:10.970820 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:10.970820 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:10.970820 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:10.972000 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:10.972000 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:10.972000 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:10.972000 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:10.972000 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:10.972000 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:10.972000 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:10.972000 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:10.972000 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:10.972000 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:10.972000 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:10.972000 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:10.972000 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:10.972000 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:10.972000 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:10.972000 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:10.972941 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:10.972941 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:10.972941 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:10.972941 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:10.972941 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:10.972941 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:10.972941 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:10.972941 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:10.972941 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:10.972941 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:10.972941 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:05:10.972941 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:10.974962 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 97/187] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:05:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:10.976333 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5748df-d012-4cb5-951b-b7de4db0acc1 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:10.976762 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5748df-d012-4cb5-951b-b7de4db0acc1 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:10.977188 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5748df-d012-4cb5-951b-b7de4db0acc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:10.999938 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-bf5748df-d012-4cb5-951b-b7de4db0acc1 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:11.001634 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-bf5748df-d012-4cb5-951b-b7de4db0acc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:11.001982 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:11.006396 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:11.008252 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604fe11-d255-41f9-a2bf-169f76484c37 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:11.057174 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6440edc0-765f-4065-a16b-a15090a9bc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:11.057946 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6440edc0-765f-4065-a16b-a15090a9bc98 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:11.058181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6440edc0-765f-4065-a16b-a15090a9bc98 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:11.058461 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6440edc0-765f-4065-a16b-a15090a9bc98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:11.059213 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6440edc0-765f-4065-a16b-a15090a9bc98 None admin] RBAC: Adding query filter params (role.id=ede2986592354e9888dde3123b2d8c82, scope.project.id=644903936a194df991e9309a715f6877, user.id=10881ece7d184527a88b151aac434c0c) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:11.059515 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6440edc0-765f-4065-a16b-a15090a9bc98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:11.061126 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6440edc0-765f-4065-a16b-a15090a9bc98 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:11.070160 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 91/188] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:05:10 2022] GET /identity/v3/role_assignments?user.id=10881ece7d184527a88b151aac434c0c&scope.project.id=644903936a194df991e9309a715f6877&role.id=ede2986592354e9888dde3123b2d8c82 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:11.070843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6440edc0-765f-4065-a16b-a15090a9bc98 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:11.095465 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 98/189] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:11.096797 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5748df-d012-4cb5-951b-b7de4db0acc1 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:11.230702 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6440edc0-765f-4065-a16b-a15090a9bc98 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:11.251146 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5748df-d012-4cb5-951b-b7de4db0acc1 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:11.270321 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df42975-abcd-443e-99d8-1c104dfcb7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:11.270956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df42975-abcd-443e-99d8-1c104dfcb7d7 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:11.271231 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df42975-abcd-443e-99d8-1c104dfcb7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:11.271551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df42975-abcd-443e-99d8-1c104dfcb7d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:11.272674 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df42975-abcd-443e-99d8-1c104dfcb7d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:11.274114 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df42975-abcd-443e-99d8-1c104dfcb7d7 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:11.275769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 92/190] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:05:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:11.276562 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df42975-abcd-443e-99d8-1c104dfcb7d7 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:11.297301 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6712de-fcc7-4aac-8c4f-cc7f08c2e710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:11.298026 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6712de-fcc7-4aac-8c4f-cc7f08c2e710 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:11.298308 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6712de-fcc7-4aac-8c4f-cc7f08c2e710 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:11.298576 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6712de-fcc7-4aac-8c4f-cc7f08c2e710 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:11.299248 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6712de-fcc7-4aac-8c4f-cc7f08c2e710 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:11.299543 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6712de-fcc7-4aac-8c4f-cc7f08c2e710 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:11.301065 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6712de-fcc7-4aac-8c4f-cc7f08c2e710 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:11.309701 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 99/191] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:11.310501 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6712de-fcc7-4aac-8c4f-cc7f08c2e710 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:11.320975 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:11.321823 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:11.413991 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df42975-abcd-443e-99d8-1c104dfcb7d7 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:11.429138 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6712de-fcc7-4aac-8c4f-cc7f08c2e710 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:11.448787 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c28abc-669d-4057-989a-d6c379746968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:11.449427 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c28abc-669d-4057-989a-d6c379746968 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:11.449706 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c28abc-669d-4057-989a-d6c379746968 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:11.449981 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c28abc-669d-4057-989a-d6c379746968 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:11.459995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c28abc-669d-4057-989a-d6c379746968 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:11.462145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00594f-24d2-4092-8f42-ac29c24d65f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:11.462748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00594f-24d2-4092-8f42-ac29c24d65f7 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:11.463024 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00594f-24d2-4092-8f42-ac29c24d65f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:11.463293 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00594f-24d2-4092-8f42-ac29c24d65f7 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:11.463978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e00594f-24d2-4092-8f42-ac29c24d65f7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:11.465281 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-48c28abc-669d-4057-989a-d6c379746968 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:05:11.465281 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:11.465281 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:11.465281 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:11.465281 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:11.465281 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:11.465281 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:11.465281 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:11.465281 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:11.465281 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:11.465281 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:11.465281 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:11.465281 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:11.465281 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:11.465281 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:11.465281 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:11.466224 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:11.466224 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:11.466224 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:11.466224 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:11.466224 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:11.466224 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:11.466224 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:11.466224 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:11.466224 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:11.466224 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:11.466224 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:11.466224 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:11.466224 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:11.466224 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:11.466224 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:11.466224 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:11.467013 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:11.467013 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:11.467013 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:11.467013 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:11.467013 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:11.467013 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:11.467013 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:11.467013 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:11.467013 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:11.467013 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:11.467013 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:05:11.467013 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:11.467676 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e00594f-24d2-4092-8f42-ac29c24d65f7 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:11.467745 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 93/192] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Sep 5 12:05:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:11.467745 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c28abc-669d-4057-989a-d6c379746968 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:11.486887 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 100/193] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Sep 5 12:05:11 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:05:11.487679 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00594f-24d2-4092-8f42-ac29c24d65f7 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:11.570878 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:11.588756 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:11.599749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c28abc-669d-4057-989a-d6c379746968 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:11.626667 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00594f-24d2-4092-8f42-ac29c24d65f7 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:11.643448 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb745e3-492e-4582-b59c-efd9577029f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:11.644113 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb745e3-492e-4582-b59c-efd9577029f2 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:11.644392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb745e3-492e-4582-b59c-efd9577029f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:11.644659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb745e3-492e-4582-b59c-efd9577029f2 None admin] PATH_INFO: `/v3/users/3b7c35d7330343e0a7a4458dbfa44989` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:11.661427 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb745e3-492e-4582-b59c-efd9577029f2 None admin] RBAC: Authorizing `identity:get_user(user_id=3b7c35d7330343e0a7a4458dbfa44989)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:11.662894 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb745e3-492e-4582-b59c-efd9577029f2 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:11.664770 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 94/194] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:05:11 2022] GET /identity/v3/users/3b7c35d7330343e0a7a4458dbfa44989 => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:11.665387 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb745e3-492e-4582-b59c-efd9577029f2 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:11.674526 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3f1c1-1a39-40b6-80be-977a0d1d6ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:11.675177 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3f1c1-1a39-40b6-80be-977a0d1d6ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:11.675457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3f1c1-1a39-40b6-80be-977a0d1d6ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:11.675760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3f1c1-1a39-40b6-80be-977a0d1d6ac8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:11.687408 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d3f1c1-1a39-40b6-80be-977a0d1d6ac8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:11.692854 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-17d3f1c1-1a39-40b6-80be-977a0d1d6ac8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:05:11.692854 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:11.692854 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:11.692854 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:11.692854 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:11.692854 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:11.692854 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:11.692854 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:11.692854 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:11.692854 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:11.692854 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:11.692854 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:11.692854 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:11.692854 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:11.692854 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:11.692854 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:11.693749 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:11.693749 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:11.693749 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:11.693749 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:11.693749 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:11.693749 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:11.693749 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:11.693749 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:11.693749 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:11.693749 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:11.693749 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:11.693749 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:11.693749 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:11.693749 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:11.693749 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:11.693749 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:11.694608 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:11.694608 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:11.694608 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:11.694608 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:11.694608 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:11.694608 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:11.694608 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:11.694608 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:11.694608 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:11.694608 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:11.694608 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:05:11.694608 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:11.699438 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 101/195] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:11.700328 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3f1c1-1a39-40b6-80be-977a0d1d6ac8 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:11.791744 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb745e3-492e-4582-b59c-efd9577029f2 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:11.838813 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3f1c1-1a39-40b6-80be-977a0d1d6ac8 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:11.844809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d81756-2331-45b7-abf3-5afc4dbd2242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:11.845687 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d81756-2331-45b7-abf3-5afc4dbd2242 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:11.846196 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d81756-2331-45b7-abf3-5afc4dbd2242 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:11.846674 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d81756-2331-45b7-abf3-5afc4dbd2242 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:11.848111 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d81756-2331-45b7-abf3-5afc4dbd2242 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:11.848635 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d81756-2331-45b7-abf3-5afc4dbd2242 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:11.851070 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d81756-2331-45b7-abf3-5afc4dbd2242 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:11.857890 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:11.864782 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 95/196] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:05:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:11.866225 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d81756-2331-45b7-abf3-5afc4dbd2242 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:11.882093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeeecfc-a8f4-4f48-b73f-dd6e415dbea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:11.882736 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeeecfc-a8f4-4f48-b73f-dd6e415dbea9 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:11.883017 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeeecfc-a8f4-4f48-b73f-dd6e415dbea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:11.883292 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeeecfc-a8f4-4f48-b73f-dd6e415dbea9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:11.888410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:11.961942 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeeecfc-a8f4-4f48-b73f-dd6e415dbea9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:11.963438 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeeecfc-a8f4-4f48-b73f-dd6e415dbea9 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:12.015252 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d81756-2331-45b7-abf3-5afc4dbd2242 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:12.054390 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357f874-a467-48e9-880f-023fa27eb070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:12.054920 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357f874-a467-48e9-880f-023fa27eb070 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:12.055216 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357f874-a467-48e9-880f-023fa27eb070 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:12.055500 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357f874-a467-48e9-880f-023fa27eb070 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:12.056227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4357f874-a467-48e9-880f-023fa27eb070 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:12.056595 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4357f874-a467-48e9-880f-023fa27eb070 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:12.058055 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4357f874-a467-48e9-880f-023fa27eb070 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:12.067335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 96/197] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Sep 5 12:05:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:12.068421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357f874-a467-48e9-880f-023fa27eb070 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:12.099423 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 102/198] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:05:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:05:12.100647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeeecfc-a8f4-4f48-b73f-dd6e415dbea9 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:12.124499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:12.186140 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:12.218453 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357f874-a467-48e9-880f-023fa27eb070 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:12.237528 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeeecfc-a8f4-4f48-b73f-dd6e415dbea9 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:12.259288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63c2d0-00f2-4c20-b2a6-f85e6f0dba96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:12.260045 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63c2d0-00f2-4c20-b2a6-f85e6f0dba96 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:12.260360 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63c2d0-00f2-4c20-b2a6-f85e6f0dba96 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:12.260568 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63c2d0-00f2-4c20-b2a6-f85e6f0dba96 None admin] PATH_INFO: `/v3/users/3b7c35d7330343e0a7a4458dbfa44989` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:12.261994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63c2d0-00f2-4c20-b2a6-f85e6f0dba96 None admin] RBAC: Authorizing `identity:get_user(user_id=3b7c35d7330343e0a7a4458dbfa44989)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:12.263162 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63c2d0-00f2-4c20-b2a6-f85e6f0dba96 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:12.265348 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 97/199] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:05:12 2022] GET /identity/v3/users/3b7c35d7330343e0a7a4458dbfa44989 => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:12.270945 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63c2d0-00f2-4c20-b2a6-f85e6f0dba96 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:12.288607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52cefff-94cc-4d9b-8a67-8af8c6826112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:12.290107 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52cefff-94cc-4d9b-8a67-8af8c6826112 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:12.290855 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52cefff-94cc-4d9b-8a67-8af8c6826112 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:12.291324 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52cefff-94cc-4d9b-8a67-8af8c6826112 None admin] PATH_INFO: `/v3/users/d39ae49df8b9460da658453f5e30dc67` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:12.293356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52cefff-94cc-4d9b-8a67-8af8c6826112 None admin] RBAC: Authorizing `identity:get_user(user_id=d39ae49df8b9460da658453f5e30dc67)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:12.295116 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52cefff-94cc-4d9b-8a67-8af8c6826112 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:12.297892 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 103/200] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:05:12 2022] GET /identity/v3/users/d39ae49df8b9460da658453f5e30dc67 => generated 289 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:12.298890 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52cefff-94cc-4d9b-8a67-8af8c6826112 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:12.391034 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63c2d0-00f2-4c20-b2a6-f85e6f0dba96 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:12.424241 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:12.430741 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bc232-0010-486a-bed0-f41db0b3d58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:12.431753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bc232-0010-486a-bed0-f41db0b3d58e None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:12.432256 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bc232-0010-486a-bed0-f41db0b3d58e None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:12.432729 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bc232-0010-486a-bed0-f41db0b3d58e None admin] PATH_INFO: `/v3/projects/75434cc693884f639e8b84015d47cc58` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:12.434335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2bc232-0010-486a-bed0-f41db0b3d58e None admin] RBAC: Authorizing `identity:get_project(project_id=75434cc693884f639e8b84015d47cc58)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:12.436130 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2bc232-0010-486a-bed0-f41db0b3d58e None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:12.442966 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52cefff-94cc-4d9b-8a67-8af8c6826112 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:12.443328 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 98/201] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:12 2022] GET /identity/v3/projects/75434cc693884f639e8b84015d47cc58 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:12.444466 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bc232-0010-486a-bed0-f41db0b3d58e None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:12.459182 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:12.490362 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174f58d-40a1-4458-b78f-9df641a513f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:12.490980 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174f58d-40a1-4458-b78f-9df641a513f9 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:12.491256 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174f58d-40a1-4458-b78f-9df641a513f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:12.491552 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174f58d-40a1-4458-b78f-9df641a513f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:12.576004 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bc232-0010-486a-bed0-f41db0b3d58e None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:12.584258 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c174f58d-40a1-4458-b78f-9df641a513f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:12.587764 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c174f58d-40a1-4458-b78f-9df641a513f9 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:12.619915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7547114-b84d-4b73-bee0-f61cc3b4f6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:12.621086 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7547114-b84d-4b73-bee0-f61cc3b4f6b1 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:12.621853 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7547114-b84d-4b73-bee0-f61cc3b4f6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:12.622882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7547114-b84d-4b73-bee0-f61cc3b4f6b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:12.693335 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:12.713090 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 104/202] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:05:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:05:12.714420 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174f58d-40a1-4458-b78f-9df641a513f9 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:12.721830 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7547114-b84d-4b73-bee0-f61cc3b4f6b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:12.723492 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7547114-b84d-4b73-bee0-f61cc3b4f6b1 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:12.737894 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:12.826426 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174f58d-40a1-4458-b78f-9df641a513f9 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:12.854708 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 99/203] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:05:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:05:12.855616 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7547114-b84d-4b73-bee0-f61cc3b4f6b1 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:12.864485 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e91d6c-f814-497d-9a5c-dde02c7e89a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:12.865448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e91d6c-f814-497d-9a5c-dde02c7e89a2 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:12.865931 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e91d6c-f814-497d-9a5c-dde02c7e89a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:12.866404 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e91d6c-f814-497d-9a5c-dde02c7e89a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:12.867354 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e91d6c-f814-497d-9a5c-dde02c7e89a2 None admin] RBAC: Adding query filter params (role.id=618c48706f7f445abeddf7eb2f8e927a, scope.project.id=75434cc693884f639e8b84015d47cc58, user.id=3b7c35d7330343e0a7a4458dbfa44989) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:12.867900 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e91d6c-f814-497d-9a5c-dde02c7e89a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:12.869539 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e91d6c-f814-497d-9a5c-dde02c7e89a2 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:12.876198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 105/204] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:05:12 2022] GET /identity/v3/role_assignments?user.id=3b7c35d7330343e0a7a4458dbfa44989&scope.project.id=75434cc693884f639e8b84015d47cc58&role.id=618c48706f7f445abeddf7eb2f8e927a => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:12.877453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e91d6c-f814-497d-9a5c-dde02c7e89a2 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:12.934839 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:12.989864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7547114-b84d-4b73-bee0-f61cc3b4f6b1 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:12.996772 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e91d6c-f814-497d-9a5c-dde02c7e89a2 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:13.010216 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:13.038245 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8060a-a895-486a-a7c9-4842aafc2231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:13.039691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8060a-a895-486a-a7c9-4842aafc2231 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:13.040215 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8060a-a895-486a-a7c9-4842aafc2231 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:13.040988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8060a-a895-486a-a7c9-4842aafc2231 None admin] PATH_INFO: `/v3/projects/405d2cd2fe68456ca034a57f24f6b015` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:13.043385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8060a-a895-486a-a7c9-4842aafc2231 None admin] RBAC: Authorizing `identity:get_project(project_id=405d2cd2fe68456ca034a57f24f6b015)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:13.045362 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8060a-a895-486a-a7c9-4842aafc2231 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:13.048002 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 106/205] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:12 2022] GET /identity/v3/projects/405d2cd2fe68456ca034a57f24f6b015 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:13.054462 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8060a-a895-486a-a7c9-4842aafc2231 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:13.057771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1abfe-9fcb-4e2f-b414-a563a1044674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:13.058747 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1abfe-9fcb-4e2f-b414-a563a1044674 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:13.059167 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1abfe-9fcb-4e2f-b414-a563a1044674 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:13.059644 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1abfe-9fcb-4e2f-b414-a563a1044674 None admin] PATH_INFO: `/v3/projects/75434cc693884f639e8b84015d47cc58` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:13.062905 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada1abfe-9fcb-4e2f-b414-a563a1044674 None admin] RBAC: Authorizing `identity:get_project(project_id=75434cc693884f639e8b84015d47cc58)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:13.065486 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada1abfe-9fcb-4e2f-b414-a563a1044674 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:13.068626 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 100/206] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:12 2022] GET /identity/v3/projects/75434cc693884f639e8b84015d47cc58 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:13.077770 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1abfe-9fcb-4e2f-b414-a563a1044674 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:13.190074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8060a-a895-486a-a7c9-4842aafc2231 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:13.214212 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:13.239223 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1abfe-9fcb-4e2f-b414-a563a1044674 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:13.240491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e66eb2f-6199-4b4f-ad23-8ecde4a0510b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:13.241237 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66eb2f-6199-4b4f-ad23-8ecde4a0510b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:13.241664 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66eb2f-6199-4b4f-ad23-8ecde4a0510b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:13.242009 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66eb2f-6199-4b4f-ad23-8ecde4a0510b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:13.245482 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e66eb2f-6199-4b4f-ad23-8ecde4a0510b None admin] RBAC: Adding query filter params (role.id=290929f17e764f019e358e7751365fd5, scope.project.id=405d2cd2fe68456ca034a57f24f6b015, user.id=d39ae49df8b9460da658453f5e30dc67) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:13.245913 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e66eb2f-6199-4b4f-ad23-8ecde4a0510b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:13.247474 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e66eb2f-6199-4b4f-ad23-8ecde4a0510b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:13.255184 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 107/207] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:05:13 2022] GET /identity/v3/role_assignments?user.id=d39ae49df8b9460da658453f5e30dc67&scope.project.id=405d2cd2fe68456ca034a57f24f6b015&role.id=290929f17e764f019e358e7751365fd5 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:13.286167 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9901905-d507-40c8-8ff4-fa4d48f36a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:13.287053 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9901905-d507-40c8-8ff4-fa4d48f36a07 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:13.287483 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9901905-d507-40c8-8ff4-fa4d48f36a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:13.287923 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9901905-d507-40c8-8ff4-fa4d48f36a07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:13.288802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9901905-d507-40c8-8ff4-fa4d48f36a07 None admin] RBAC: Adding query filter params (role.id=20bf8c77837d43b590a2bd9a67d01f84, scope.project.id=75434cc693884f639e8b84015d47cc58, user.id=3b7c35d7330343e0a7a4458dbfa44989) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:13.289253 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9901905-d507-40c8-8ff4-fa4d48f36a07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:13.290861 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9901905-d507-40c8-8ff4-fa4d48f36a07 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:13.298072 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 101/208] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:05:13 2022] GET /identity/v3/role_assignments?user.id=3b7c35d7330343e0a7a4458dbfa44989&scope.project.id=75434cc693884f639e8b84015d47cc58&role.id=20bf8c77837d43b590a2bd9a67d01f84 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:13.568745 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cabe3-a274-4955-86f2-2134cb80e09a None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:13.569140 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cabe3-a274-4955-86f2-2134cb80e09a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:13.569340 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cabe3-a274-4955-86f2-2134cb80e09a None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:13.570773 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 108/209] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:13.576299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610c47b3-ba13-4f85-ac20-3c6470c6c7d0 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:13.576843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610c47b3-ba13-4f85-ac20-3c6470c6c7d0 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:13.577359 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610c47b3-ba13-4f85-ac20-3c6470c6c7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:13.598024 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-610c47b3-ba13-4f85-ac20-3c6470c6c7d0 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:13.599575 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-610c47b3-ba13-4f85-ac20-3c6470c6c7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:13.648634 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:13.686869 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 102/210] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:13.851384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36aa5e-147b-42f1-aa53-b9ef9dd4ffc6 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:13.851716 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36aa5e-147b-42f1-aa53-b9ef9dd4ffc6 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:13.851904 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36aa5e-147b-42f1-aa53-b9ef9dd4ffc6 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:13.853369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 109/211] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:13.858764 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ea237-23bf-4af7-a7e2-f968cfd87a7f None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:13.859628 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ea237-23bf-4af7-a7e2-f968cfd87a7f None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:13.860117 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ea237-23bf-4af7-a7e2-f968cfd87a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:13.879492 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-cb1ea237-23bf-4af7-a7e2-f968cfd87a7f None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:13.881045 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-cb1ea237-23bf-4af7-a7e2-f968cfd87a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:13.956166 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 103/212] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:05:13.978318 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36aa5e-147b-42f1-aa53-b9ef9dd4ffc6 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:14.090245 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36aa5e-147b-42f1-aa53-b9ef9dd4ffc6 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:14.126524 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f00e4-1c76-44b4-866e-fd3558c15104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:14.127081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f00e4-1c76-44b4-866e-fd3558c15104 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:14.127355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f00e4-1c76-44b4-866e-fd3558c15104 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:14.127655 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f00e4-1c76-44b4-866e-fd3558c15104 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:14.142898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f00e4-1c76-44b4-866e-fd3558c15104 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:14.147898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-a29f00e4-1c76-44b4-866e-fd3558c15104 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:14.147898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:14.147898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:14.147898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:14.147898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:14.147898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:14.147898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:14.147898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:14.147898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:14.147898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:14.147898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:14.147898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:14.147898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:14.147898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:14.147898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:14.147898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:14.148773 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:14.148773 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:14.148773 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:14.148773 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:14.148773 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:14.148773 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:14.148773 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:14.148773 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:14.148773 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:14.148773 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:14.148773 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:14.148773 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:14.148773 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:14.148773 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:14.148773 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:14.148773 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:14.149598 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:14.149598 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:14.149598 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:14.149598 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:14.149598 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:14.149598 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:14.149598 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:14.149598 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:14.149598 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:14.149598 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:14.149598 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:14.149598 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:14.149598 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 110/213] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:14.156620 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ea237-23bf-4af7-a7e2-f968cfd87a7f None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:14.232957 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f00e4-1c76-44b4-866e-fd3558c15104 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:14.261913 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ea237-23bf-4af7-a7e2-f968cfd87a7f None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:14.263904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:14.286272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:14.302614 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae477db7-d402-4901-95ea-c28eb5c3f91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:14.303236 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae477db7-d402-4901-95ea-c28eb5c3f91b None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:14.303564 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae477db7-d402-4901-95ea-c28eb5c3f91b None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:14.303819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae477db7-d402-4901-95ea-c28eb5c3f91b None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:14.304672 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae477db7-d402-4901-95ea-c28eb5c3f91b None admin] RBAC: Adding query filter params (name=member) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:14.304980 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae477db7-d402-4901-95ea-c28eb5c3f91b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:14.307084 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae477db7-d402-4901-95ea-c28eb5c3f91b None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:14.318715 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 104/214] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Sep 5 12:05:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:14.328747 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae477db7-d402-4901-95ea-c28eb5c3f91b None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:14.356802 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f00e4-1c76-44b4-866e-fd3558c15104 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:14.394126 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbea684-2e9a-406b-9391-f946fafecfe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:14.394841 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbea684-2e9a-406b-9391-f946fafecfe1 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:14.395222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbea684-2e9a-406b-9391-f946fafecfe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:14.395621 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbea684-2e9a-406b-9391-f946fafecfe1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:14.396838 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbea684-2e9a-406b-9391-f946fafecfe1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:14.398431 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbea684-2e9a-406b-9391-f946fafecfe1 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:14.400414 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 111/215] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:05:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:14.405868 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbea684-2e9a-406b-9391-f946fafecfe1 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:14.446639 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae477db7-d402-4901-95ea-c28eb5c3f91b None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:14.496096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e26326-1057-4bf9-bf83-f924a52c6aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:14.497277 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e26326-1057-4bf9-bf83-f924a52c6aed None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:14.497736 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e26326-1057-4bf9-bf83-f924a52c6aed None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:14.498476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e26326-1057-4bf9-bf83-f924a52c6aed None admin] PATH_INFO: `/v3/users/0d4d9ce0bb284c378ab8627e107c20a5` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:14.518849 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e26326-1057-4bf9-bf83-f924a52c6aed None admin] RBAC: Authorizing `identity:get_user(user_id=0d4d9ce0bb284c378ab8627e107c20a5)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:14.520913 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e26326-1057-4bf9-bf83-f924a52c6aed None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:14.521593 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbea684-2e9a-406b-9391-f946fafecfe1 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:14.527953 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 105/216] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:05:14 2022] GET /identity/v3/users/0d4d9ce0bb284c378ab8627e107c20a5 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:14.529281 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e26326-1057-4bf9-bf83-f924a52c6aed None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:14.539959 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:14.564317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:14.572695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab9d14-5f64-4a9a-a126-4b704d8ab21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:14.573355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab9d14-5f64-4a9a-a126-4b704d8ab21b None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:14.573633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab9d14-5f64-4a9a-a126-4b704d8ab21b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:14.573905 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab9d14-5f64-4a9a-a126-4b704d8ab21b None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:14.575221 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bab9d14-5f64-4a9a-a126-4b704d8ab21b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:14.578158 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bab9d14-5f64-4a9a-a126-4b704d8ab21b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:14.607811 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 112/217] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Sep 5 12:05:14 2022] POST /identity/v3/users => generated 297 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:05:14.669625 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e26326-1057-4bf9-bf83-f924a52c6aed None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:14.712973 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e7e46-f526-4298-8853-34cc27cee08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:14.714068 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e7e46-f526-4298-8853-34cc27cee08a None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:14.714548 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e7e46-f526-4298-8853-34cc27cee08a None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:14.715016 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e7e46-f526-4298-8853-34cc27cee08a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:14.793625 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e7e46-f526-4298-8853-34cc27cee08a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:14.795490 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e7e46-f526-4298-8853-34cc27cee08a None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:14.822502 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:14.907868 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 106/218] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:05:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 05 12:05:14.913546 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab9d14-5f64-4a9a-a126-4b704d8ab21b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:15.061891 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab9d14-5f64-4a9a-a126-4b704d8ab21b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:15.130744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d4cec-2024-4fe2-a540-99bd6998b5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:15.133125 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d4cec-2024-4fe2-a540-99bd6998b5b9 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:15.133890 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d4cec-2024-4fe2-a540-99bd6998b5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:15.134619 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d4cec-2024-4fe2-a540-99bd6998b5b9 None admin] PATH_INFO: `/v3/projects/75434cc693884f639e8b84015d47cc58` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:15.140350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493d4cec-2024-4fe2-a540-99bd6998b5b9 None admin] RBAC: Authorizing `identity:get_project(project_id=75434cc693884f639e8b84015d47cc58)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:15.143627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493d4cec-2024-4fe2-a540-99bd6998b5b9 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:15.146988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 113/219] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:14 2022] GET /identity/v3/projects/75434cc693884f639e8b84015d47cc58 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:15.153848 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e7e46-f526-4298-8853-34cc27cee08a None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:15.214426 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f1a3e-d38f-4512-b8be-976700bd2f5c None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:15.215356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f1a3e-d38f-4512-b8be-976700bd2f5c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:15.216164 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f1a3e-d38f-4512-b8be-976700bd2f5c None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:15.218913 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 114/220] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:15.225386 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090a3c0-99c4-4df3-80b0-690545750940 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:15.227472 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090a3c0-99c4-4df3-80b0-690545750940 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:15.228279 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090a3c0-99c4-4df3-80b0-690545750940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:15.281044 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-a090a3c0-99c4-4df3-80b0-690545750940 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:15.287291 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a090a3c0-99c4-4df3-80b0-690545750940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:15.362594 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e7e46-f526-4298-8853-34cc27cee08a None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:15.379398 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:15.390117 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:15.409594 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820d329-2017-4c14-8ad2-ef6b74ad8e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:15.410214 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820d329-2017-4c14-8ad2-ef6b74ad8e2f None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:15.410494 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820d329-2017-4c14-8ad2-ef6b74ad8e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:15.410764 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820d329-2017-4c14-8ad2-ef6b74ad8e2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:15.411502 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4820d329-2017-4c14-8ad2-ef6b74ad8e2f None admin] RBAC: Adding query filter params (role.id=ede2986592354e9888dde3123b2d8c82, scope.project.id=75434cc693884f639e8b84015d47cc58, user.id=0d4d9ce0bb284c378ab8627e107c20a5) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:15.411831 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4820d329-2017-4c14-8ad2-ef6b74ad8e2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:15.413265 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4820d329-2017-4c14-8ad2-ef6b74ad8e2f None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:15.432090 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 107/221] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:05:15 2022] GET /identity/v3/role_assignments?user.id=0d4d9ce0bb284c378ab8627e107c20a5&scope.project.id=75434cc693884f639e8b84015d47cc58&role.id=ede2986592354e9888dde3123b2d8c82 => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:15.443604 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 115/222] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:05:15.689720 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ed82a-b2ae-411f-9d28-a55db0d93bb4 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:15.689912 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ed82a-b2ae-411f-9d28-a55db0d93bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:15.690176 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ed82a-b2ae-411f-9d28-a55db0d93bb4 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:15.691712 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 108/223] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:15.697287 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09facf-5f7b-461f-8e3e-fac465d0fdc6 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:15.697580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09facf-5f7b-461f-8e3e-fac465d0fdc6 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:15.697852 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09facf-5f7b-461f-8e3e-fac465d0fdc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:15.717347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-8b09facf-5f7b-461f-8e3e-fac465d0fdc6 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:15.719071 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8b09facf-5f7b-461f-8e3e-fac465d0fdc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:15.792212 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 116/224] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:05:15.814792 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ed82a-b2ae-411f-9d28-a55db0d93bb4 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:15.928828 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ed82a-b2ae-411f-9d28-a55db0d93bb4 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:15.965381 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a363887-3f83-43d9-84ff-01a1e8fe4046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:15.965943 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a363887-3f83-43d9-84ff-01a1e8fe4046 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:15.966221 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a363887-3f83-43d9-84ff-01a1e8fe4046 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:15.966494 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a363887-3f83-43d9-84ff-01a1e8fe4046 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:15.977985 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a363887-3f83-43d9-84ff-01a1e8fe4046 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:15.984043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-6a363887-3f83-43d9-84ff-01a1e8fe4046 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:05:15.984043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:15.984043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:15.984043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:15.984043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:15.984043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:15.984043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:15.984043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:15.984043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:15.984043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:15.984043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:15.984043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:15.984043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:15.984043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:15.984043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:15.984043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:15.984956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:15.984956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:15.984956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:15.984956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:15.984956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:15.984956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:15.984956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:15.984956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:15.984956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:15.984956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:15.984956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:15.984956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:15.984956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:15.984956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:15.984956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:15.984956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:15.986359 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:15.986359 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:15.986359 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:15.986359 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:15.986359 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:15.986359 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:15.986359 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:15.986359 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:15.986359 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:15.986359 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:15.986359 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:05:15.986359 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:15.986359 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 109/225] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:05:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:15.991138 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09facf-5f7b-461f-8e3e-fac465d0fdc6 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:16.100454 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a363887-3f83-43d9-84ff-01a1e8fe4046 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:16.102484 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:16.113162 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09facf-5f7b-461f-8e3e-fac465d0fdc6 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:16.171934 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:16.190377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f8e34a-73b8-43d1-a713-775cc359c53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:16.191111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8e34a-73b8-43d1-a713-775cc359c53a None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:16.191260 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8e34a-73b8-43d1-a713-775cc359c53a None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:16.191658 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8e34a-73b8-43d1-a713-775cc359c53a None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:16.192344 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f8e34a-73b8-43d1-a713-775cc359c53a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:16.192808 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f8e34a-73b8-43d1-a713-775cc359c53a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:16.194317 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f8e34a-73b8-43d1-a713-775cc359c53a None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:16.213692 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 117/226] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:16.224725 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f8e34a-73b8-43d1-a713-775cc359c53a None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:16.259166 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a363887-3f83-43d9-84ff-01a1e8fe4046 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:16.301173 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221a91e-8abc-4aff-affa-8b3312e9b863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:16.301843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221a91e-8abc-4aff-affa-8b3312e9b863 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:16.302123 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221a91e-8abc-4aff-affa-8b3312e9b863 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:16.302399 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221a91e-8abc-4aff-affa-8b3312e9b863 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:16.316066 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e221a91e-8abc-4aff-affa-8b3312e9b863 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:16.322732 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-e221a91e-8abc-4aff-affa-8b3312e9b863 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:05:16.322732 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:16.322732 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:16.322732 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:16.322732 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:16.322732 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:16.322732 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:16.322732 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:16.322732 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:16.322732 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:16.322732 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:16.322732 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:16.322732 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:16.322732 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:16.322732 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:16.322732 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:16.324127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:16.324127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:16.324127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:16.324127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:16.324127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:16.324127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:16.324127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:16.324127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:16.324127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:16.324127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:16.324127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:16.324127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:16.324127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:16.324127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:16.324127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:16.324127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:16.325335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:16.325335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:16.325335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:16.325335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:16.325335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:16.325335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:16.325335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:16.325335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:16.325335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:16.325335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:16.325335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:05:16.325335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:16.325335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 110/227] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Sep 5 12:05:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:05:16.333609 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221a91e-8abc-4aff-affa-8b3312e9b863 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:16.489538 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221a91e-8abc-4aff-affa-8b3312e9b863 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:16.497559 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:16.513709 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f8e34a-73b8-43d1-a713-775cc359c53a None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:16.537402 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b0c8f-2e66-4146-9942-eea0962eebe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:16.538017 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b0c8f-2e66-4146-9942-eea0962eebe1 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:16.538296 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b0c8f-2e66-4146-9942-eea0962eebe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:16.538566 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b0c8f-2e66-4146-9942-eea0962eebe1 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:16.539247 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b0c8f-2e66-4146-9942-eea0962eebe1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:16.539564 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b0c8f-2e66-4146-9942-eea0962eebe1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:16.545068 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b0c8f-2e66-4146-9942-eea0962eebe1 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:16.569728 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 111/228] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:05:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:16.575686 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b0c8f-2e66-4146-9942-eea0962eebe1 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:16.577015 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2141bcd-9cc0-4740-b66d-58b8aa46f4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:16.582712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2141bcd-9cc0-4740-b66d-58b8aa46f4ac None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:16.582712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2141bcd-9cc0-4740-b66d-58b8aa46f4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:16.582712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2141bcd-9cc0-4740-b66d-58b8aa46f4ac None admin] PATH_INFO: `/v3/users/3b7c35d7330343e0a7a4458dbfa44989` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:16.582712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2141bcd-9cc0-4740-b66d-58b8aa46f4ac None admin] RBAC: Authorizing `identity:get_user(user_id=3b7c35d7330343e0a7a4458dbfa44989)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:16.582712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2141bcd-9cc0-4740-b66d-58b8aa46f4ac None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:16.582712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 118/229] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:05:16 2022] GET /identity/v3/users/3b7c35d7330343e0a7a4458dbfa44989 => generated 283 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:16.588129 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2141bcd-9cc0-4740-b66d-58b8aa46f4ac None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:16.625020 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:16.719054 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b0c8f-2e66-4146-9942-eea0962eebe1 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:16.742987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2141bcd-9cc0-4740-b66d-58b8aa46f4ac None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:16.788982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f27ca8-4a37-4bbc-b109-a7331d5bad21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:16.789978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f27ca8-4a37-4bbc-b109-a7331d5bad21 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:16.790407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f27ca8-4a37-4bbc-b109-a7331d5bad21 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:16.790841 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f27ca8-4a37-4bbc-b109-a7331d5bad21 None admin] PATH_INFO: `/v3/projects/75434cc693884f639e8b84015d47cc58` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:16.792449 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f27ca8-4a37-4bbc-b109-a7331d5bad21 None admin] RBAC: Authorizing `identity:get_project(project_id=75434cc693884f639e8b84015d47cc58)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:16.794616 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f27ca8-4a37-4bbc-b109-a7331d5bad21 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:16.798487 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 119/230] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:16 2022] GET /identity/v3/projects/75434cc693884f639e8b84015d47cc58 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:16.801305 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eea359-f655-4ee7-a459-6e1f9b9ddeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:16.804323 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f27ca8-4a37-4bbc-b109-a7331d5bad21 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:16.804927 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eea359-f655-4ee7-a459-6e1f9b9ddeee None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:16.805859 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eea359-f655-4ee7-a459-6e1f9b9ddeee None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:16.806324 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eea359-f655-4ee7-a459-6e1f9b9ddeee None admin] PATH_INFO: `/v3/users/3b7c35d7330343e0a7a4458dbfa44989` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:16.811198 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eea359-f655-4ee7-a459-6e1f9b9ddeee None admin] RBAC: Authorizing `identity:get_user(user_id=3b7c35d7330343e0a7a4458dbfa44989)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:16.814217 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eea359-f655-4ee7-a459-6e1f9b9ddeee None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:16.817474 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 112/231] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:05:16 2022] GET /identity/v3/users/3b7c35d7330343e0a7a4458dbfa44989 => generated 283 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:16.823355 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eea359-f655-4ee7-a459-6e1f9b9ddeee None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:16.870281 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:16.967474 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eea359-f655-4ee7-a459-6e1f9b9ddeee None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:17.011993 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:17.024055 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f27ca8-4a37-4bbc-b109-a7331d5bad21 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:17.039534 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9915973e-b57a-4f0e-acd0-5a02f10ac708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:17.040164 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915973e-b57a-4f0e-acd0-5a02f10ac708 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:17.040444 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915973e-b57a-4f0e-acd0-5a02f10ac708 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:17.040718 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915973e-b57a-4f0e-acd0-5a02f10ac708 None admin] PATH_INFO: `/v3/projects/75434cc693884f639e8b84015d47cc58` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:17.043156 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9915973e-b57a-4f0e-acd0-5a02f10ac708 None admin] RBAC: Authorizing `identity:get_project(project_id=75434cc693884f639e8b84015d47cc58)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:17.044638 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9915973e-b57a-4f0e-acd0-5a02f10ac708 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:17.046544 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 113/232] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:16 2022] GET /identity/v3/projects/75434cc693884f639e8b84015d47cc58 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:17.052924 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9915973e-b57a-4f0e-acd0-5a02f10ac708 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:17.099394 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfddbcb-b9fd-4a15-8d86-6b02910e6127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:17.100409 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfddbcb-b9fd-4a15-8d86-6b02910e6127 None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:17.100897 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfddbcb-b9fd-4a15-8d86-6b02910e6127 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:17.101394 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfddbcb-b9fd-4a15-8d86-6b02910e6127 None admin] PATH_INFO: `/v3/projects/75434cc693884f639e8b84015d47cc58/users/3b7c35d7330343e0a7a4458dbfa44989/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:17.103630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfddbcb-b9fd-4a15-8d86-6b02910e6127 None admin] RBAC: Authorizing `identity:create_grant(project_id=75434cc693884f639e8b84015d47cc58, user_id=3b7c35d7330343e0a7a4458dbfa44989, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:17.105453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfddbcb-b9fd-4a15-8d86-6b02910e6127 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:17.122080 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 120/233] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Sep 5 12:05:16 2022] PUT /identity/v3/projects/75434cc693884f639e8b84015d47cc58/users/3b7c35d7330343e0a7a4458dbfa44989/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:05:17.124467 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ef6da-055a-4daa-acc5-8298f9a42d6f None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:17.125110 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ef6da-055a-4daa-acc5-8298f9a42d6f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:17.125597 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ef6da-055a-4daa-acc5-8298f9a42d6f None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:17.127258 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 121/234] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:17.132815 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69590a-a41b-402d-b820-e3bc30c35da2 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:17.133354 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69590a-a41b-402d-b820-e3bc30c35da2 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:17.133902 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69590a-a41b-402d-b820-e3bc30c35da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:17.158907 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-eb69590a-a41b-402d-b820-e3bc30c35da2 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:17.161637 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-eb69590a-a41b-402d-b820-e3bc30c35da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:17.188543 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9915973e-b57a-4f0e-acd0-5a02f10ac708 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:17.190717 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:17.262181 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 122/235] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:05:17.273020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362cbe2-4b7e-42d5-a701-ade6f3d1ba9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:17.274072 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362cbe2-4b7e-42d5-a701-ade6f3d1ba9b None admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:17.274559 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362cbe2-4b7e-42d5-a701-ade6f3d1ba9b None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:17.275056 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362cbe2-4b7e-42d5-a701-ade6f3d1ba9b None admin] PATH_INFO: `/v3/projects/75434cc693884f639e8b84015d47cc58/users/3b7c35d7330343e0a7a4458dbfa44989/roles/20bf8c77837d43b590a2bd9a67d01f84` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:17.280374 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362cbe2-4b7e-42d5-a701-ade6f3d1ba9b None admin] RBAC: Authorizing `identity:create_grant(project_id=75434cc693884f639e8b84015d47cc58, user_id=3b7c35d7330343e0a7a4458dbfa44989, role_id=20bf8c77837d43b590a2bd9a67d01f84)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:17.282189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362cbe2-4b7e-42d5-a701-ade6f3d1ba9b None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:17.306213 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 114/236] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Sep 5 12:05:17 2022] PUT /identity/v3/projects/75434cc693884f639e8b84015d47cc58/users/3b7c35d7330343e0a7a4458dbfa44989/roles/20bf8c77837d43b590a2bd9a67d01f84 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:05:17.366453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884333df-2281-4021-908a-149899b3673d None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:17.366762 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884333df-2281-4021-908a-149899b3673d None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:17.367010 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884333df-2281-4021-908a-149899b3673d None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:17.374471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 123/237] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:17.381233 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99414fc-4e61-4a63-9163-05fae8e4b581 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:17.381882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99414fc-4e61-4a63-9163-05fae8e4b581 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:17.382404 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99414fc-4e61-4a63-9163-05fae8e4b581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:17.407337 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-e99414fc-4e61-4a63-9163-05fae8e4b581 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:17.408610 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-e99414fc-4e61-4a63-9163-05fae8e4b581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:17.500920 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 115/238] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:17.556787 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884333df-2281-4021-908a-149899b3673d None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:17.654532 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:17.688535 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884333df-2281-4021-908a-149899b3673d None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:17.740212 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c6b85-2ecb-46cd-8722-85790db40c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:17.740855 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c6b85-2ecb-46cd-8722-85790db40c37 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:17.741133 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c6b85-2ecb-46cd-8722-85790db40c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:17.741413 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c6b85-2ecb-46cd-8722-85790db40c37 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:17.756309 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c6b85-2ecb-46cd-8722-85790db40c37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:17.761180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-602c6b85-2ecb-46cd-8722-85790db40c37 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:05:17.761180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:17.761180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:17.761180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:17.761180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:17.761180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:17.761180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:17.761180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:17.761180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:17.761180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:17.761180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:17.761180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:17.761180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:17.761180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:17.761180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:17.761180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:17.762082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:17.762082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:17.762082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:17.762082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:17.762082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:17.762082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:17.762082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:17.762082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:17.762082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:17.762082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:17.762082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:17.762082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:17.762082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:17.762082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:17.762082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:17.762082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:17.763331 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:17.763331 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:17.763331 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:17.763331 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:17.763331 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:17.763331 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:17.763331 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:17.763331 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:17.763331 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:17.763331 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:17.763331 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:05:17.763331 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:17.763331 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 124/239] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:17.768854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99414fc-4e61-4a63-9163-05fae8e4b581 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:17.784521 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c6b85-2ecb-46cd-8722-85790db40c37 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:17.931886 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:17.943724 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99414fc-4e61-4a63-9163-05fae8e4b581 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:17.966494 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c6b85-2ecb-46cd-8722-85790db40c37 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:18.002458 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb3861b-9249-4ebf-bb20-67d0304c51a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:18.003290 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3861b-9249-4ebf-bb20-67d0304c51a0 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:18.003595 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3861b-9249-4ebf-bb20-67d0304c51a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:18.003870 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3861b-9249-4ebf-bb20-67d0304c51a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:18.004550 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb3861b-9249-4ebf-bb20-67d0304c51a0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:18.004855 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb3861b-9249-4ebf-bb20-67d0304c51a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:18.006257 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb3861b-9249-4ebf-bb20-67d0304c51a0 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:18.023225 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 116/240] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Sep 5 12:05:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:18.028742 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520ae50-6833-46a9-b054-8ff64721b6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:18.029293 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b29b2-9242-4a3b-90bd-149a16887bac None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:18.029419 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520ae50-6833-46a9-b054-8ff64721b6ed None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:18.029595 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b29b2-9242-4a3b-90bd-149a16887bac None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:18.029688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520ae50-6833-46a9-b054-8ff64721b6ed None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:18.030378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b29b2-9242-4a3b-90bd-149a16887bac None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:18.030478 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520ae50-6833-46a9-b054-8ff64721b6ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:18.031614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520ae50-6833-46a9-b054-8ff64721b6ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:18.032174 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 117/241] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:18.032884 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b29b2-9242-4a3b-90bd-149a16887bac None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:18.035801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520ae50-6833-46a9-b054-8ff64721b6ed None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:18.037959 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 125/242] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:05:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:18.039408 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ecc98-909a-4a3c-84b4-909e50101404 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:18.039742 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ecc98-909a-4a3c-84b4-909e50101404 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:18.040022 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ecc98-909a-4a3c-84b4-909e50101404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:18.059179 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-192ecc98-909a-4a3c-84b4-909e50101404 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:18.060421 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-192ecc98-909a-4a3c-84b4-909e50101404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:18.063879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:18.147240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 126/243] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:18.148197 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ecc98-909a-4a3c-84b4-909e50101404 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:18.166606 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b29b2-9242-4a3b-90bd-149a16887bac None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:18.233611 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c543268e-0ca9-4deb-90c6-be85cd236506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:18.234164 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543268e-0ca9-4deb-90c6-be85cd236506 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:18.234442 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543268e-0ca9-4deb-90c6-be85cd236506 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:18.234717 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543268e-0ca9-4deb-90c6-be85cd236506 None admin] PATH_INFO: `/v3/users/e31a11cf45f14b89bc3c55c95dc76009` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:18.265642 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c543268e-0ca9-4deb-90c6-be85cd236506 None admin] RBAC: Authorizing `identity:get_user(user_id=e31a11cf45f14b89bc3c55c95dc76009)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:18.267121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c543268e-0ca9-4deb-90c6-be85cd236506 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:18.268852 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 118/244] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:05:18 2022] GET /identity/v3/users/e31a11cf45f14b89bc3c55c95dc76009 => generated 297 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:18.275572 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c543268e-0ca9-4deb-90c6-be85cd236506 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:18.308525 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:18.345930 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ecc98-909a-4a3c-84b4-909e50101404 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:18.399936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd9764-f6c7-4ec5-a3ba-ac32b05c8b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:18.400552 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd9764-f6c7-4ec5-a3ba-ac32b05c8b20 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:18.402910 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd9764-f6c7-4ec5-a3ba-ac32b05c8b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:18.403541 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd9764-f6c7-4ec5-a3ba-ac32b05c8b20 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:18.404213 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd9764-f6c7-4ec5-a3ba-ac32b05c8b20 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:18.407379 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd9764-f6c7-4ec5-a3ba-ac32b05c8b20 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:18.433713 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:18.456707 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 127/245] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Sep 5 12:05:18 2022] POST /identity/v3/users => generated 293 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:05:18.458951 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd9764-f6c7-4ec5-a3ba-ac32b05c8b20 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:18.459722 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c543268e-0ca9-4deb-90c6-be85cd236506 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:18.540754 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e5517-66b4-48a9-a589-c22b15b4a6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:18.541428 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e5517-66b4-48a9-a589-c22b15b4a6c4 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:18.541709 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e5517-66b4-48a9-a589-c22b15b4a6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:18.541980 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e5517-66b4-48a9-a589-c22b15b4a6c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:18.617088 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd9764-f6c7-4ec5-a3ba-ac32b05c8b20 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:18.655956 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b1f0a-50c2-4b9d-a8c2-2bd70af2c6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:18.656647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b1f0a-50c2-4b9d-a8c2-2bd70af2c6ee None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:18.656926 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b1f0a-50c2-4b9d-a8c2-2bd70af2c6ee None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:18.657335 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b1f0a-50c2-4b9d-a8c2-2bd70af2c6ee None admin] PATH_INFO: `/v3/roles/member` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:18.668838 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0b1f0a-50c2-4b9d-a8c2-2bd70af2c6ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:18.674661 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-cf0b1f0a-50c2-4b9d-a8c2-2bd70af2c6ee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:18.674661 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:18.674661 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:18.674661 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:18.674661 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:18.674661 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:18.674661 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:18.674661 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:18.674661 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:18.674661 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:18.674661 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:18.674661 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:18.674661 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:18.674661 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:18.674661 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:18.674661 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:18.675716 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:18.675716 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:18.675716 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:18.675716 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:18.675716 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:18.675716 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:18.675716 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:18.675716 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:18.675716 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:18.675716 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:18.675716 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:18.675716 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:18.675716 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:18.675716 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:18.675716 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:18.675716 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:18.676531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:18.676531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:18.676531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:18.676531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:18.676531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:18.676531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:18.676531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:18.676531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:18.676531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:18.676531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:18.676531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:18.676531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:18.676531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 128/246] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:05:18.680807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942e5517-66b4-48a9-a589-c22b15b4a6c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:18.680807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942e5517-66b4-48a9-a589-c22b15b4a6c4 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:18.684812 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b1f0a-50c2-4b9d-a8c2-2bd70af2c6ee None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:18.700779 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:18.766313 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:18.823173 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 119/247] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:05:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:05:18.826406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b1f0a-50c2-4b9d-a8c2-2bd70af2c6ee None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:18.830575 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e5517-66b4-48a9-a589-c22b15b4a6c4 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:18.871433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4e297-3ac3-4099-9041-ebc1bfed08d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:18.872477 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4e297-3ac3-4099-9041-ebc1bfed08d9 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:18.873180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4e297-3ac3-4099-9041-ebc1bfed08d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:18.874221 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4e297-3ac3-4099-9041-ebc1bfed08d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:18.876328 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4e297-3ac3-4099-9041-ebc1bfed08d9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:18.876645 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4e297-3ac3-4099-9041-ebc1bfed08d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:18.878050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4e297-3ac3-4099-9041-ebc1bfed08d9 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:18.894825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 129/248] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Sep 5 12:05:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:18.900214 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4e297-3ac3-4099-9041-ebc1bfed08d9 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:18.975573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e5517-66b4-48a9-a589-c22b15b4a6c4 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:18.994741 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:19.020885 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d10c5d-c85e-4cb4-aedc-afe7fd197769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:19.021609 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d10c5d-c85e-4cb4-aedc-afe7fd197769 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:19.022066 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d10c5d-c85e-4cb4-aedc-afe7fd197769 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:19.022348 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d10c5d-c85e-4cb4-aedc-afe7fd197769 None admin] PATH_INFO: `/v3/projects/75434cc693884f639e8b84015d47cc58` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:19.026169 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d10c5d-c85e-4cb4-aedc-afe7fd197769 None admin] RBAC: Authorizing `identity:get_project(project_id=75434cc693884f639e8b84015d47cc58)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:19.028282 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d10c5d-c85e-4cb4-aedc-afe7fd197769 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:19.030309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 120/249] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:18 2022] GET /identity/v3/projects/75434cc693884f639e8b84015d47cc58 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:19.037269 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d10c5d-c85e-4cb4-aedc-afe7fd197769 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:19.037860 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4e297-3ac3-4099-9041-ebc1bfed08d9 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:19.061123 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:19.090827 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e17adb5-73f9-429c-bf3d-0b577a96ce35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:19.091401 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17adb5-73f9-429c-bf3d-0b577a96ce35 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:19.091717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17adb5-73f9-429c-bf3d-0b577a96ce35 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:19.091995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17adb5-73f9-429c-bf3d-0b577a96ce35 None admin] PATH_INFO: `/v3/users/0d4d9ce0bb284c378ab8627e107c20a5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:19.095109 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e17adb5-73f9-429c-bf3d-0b577a96ce35 None admin] RBAC: Authorizing `identity:get_user(user_id=0d4d9ce0bb284c378ab8627e107c20a5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:19.096630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e17adb5-73f9-429c-bf3d-0b577a96ce35 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:19.099240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 130/250] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:05:18 2022] GET /identity/v3/users/0d4d9ce0bb284c378ab8627e107c20a5 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:19.105024 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e17adb5-73f9-429c-bf3d-0b577a96ce35 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:19.179481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d10c5d-c85e-4cb4-aedc-afe7fd197769 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:19.220759 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e17adb5-73f9-429c-bf3d-0b577a96ce35 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:19.229042 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa0f4aa-30a7-4e86-886c-20a710e6ddc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:19.229936 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0f4aa-30a7-4e86-886c-20a710e6ddc2 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:19.230235 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0f4aa-30a7-4e86-886c-20a710e6ddc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:19.230644 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0f4aa-30a7-4e86-886c-20a710e6ddc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:19.231494 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa0f4aa-30a7-4e86-886c-20a710e6ddc2 None admin] RBAC: Adding query filter params (role.id=290929f17e764f019e358e7751365fd5, scope.project.id=75434cc693884f639e8b84015d47cc58, user.id=e31a11cf45f14b89bc3c55c95dc76009) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:19.231834 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa0f4aa-30a7-4e86-886c-20a710e6ddc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:19.233978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa0f4aa-30a7-4e86-886c-20a710e6ddc2 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:19.240302 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 121/251] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:05:19 2022] GET /identity/v3/role_assignments?user.id=e31a11cf45f14b89bc3c55c95dc76009&scope.project.id=75434cc693884f639e8b84015d47cc58&role.id=290929f17e764f019e358e7751365fd5 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:19.267406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c178586-6268-494c-b729-d98f0b8cf5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:19.268103 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c178586-6268-494c-b729-d98f0b8cf5be None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:19.268387 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c178586-6268-494c-b729-d98f0b8cf5be None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:19.268665 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c178586-6268-494c-b729-d98f0b8cf5be None admin] PATH_INFO: `/v3/projects/75434cc693884f639e8b84015d47cc58` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:19.269933 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c178586-6268-494c-b729-d98f0b8cf5be None admin] RBAC: Authorizing `identity:get_project(project_id=75434cc693884f639e8b84015d47cc58)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:19.271560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c178586-6268-494c-b729-d98f0b8cf5be None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:19.278681 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 131/252] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:19 2022] GET /identity/v3/projects/75434cc693884f639e8b84015d47cc58 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:19.280825 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa0f4aa-30a7-4e86-886c-20a710e6ddc2 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:19.360316 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:19.471621 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa0f4aa-30a7-4e86-886c-20a710e6ddc2 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:19.517088 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad378109-602d-4ffd-8f1b-e7aff1629dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:19.517761 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad378109-602d-4ffd-8f1b-e7aff1629dbb None admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:19.518043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad378109-602d-4ffd-8f1b-e7aff1629dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:19.518319 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad378109-602d-4ffd-8f1b-e7aff1629dbb None admin] PATH_INFO: `/v3/projects/75434cc693884f639e8b84015d47cc58/users/0d4d9ce0bb284c378ab8627e107c20a5/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:19.520317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad378109-602d-4ffd-8f1b-e7aff1629dbb None admin] RBAC: Authorizing `identity:create_grant(project_id=75434cc693884f639e8b84015d47cc58, user_id=0d4d9ce0bb284c378ab8627e107c20a5, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:19.521850 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad378109-602d-4ffd-8f1b-e7aff1629dbb None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:19.539978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 122/253] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Sep 5 12:05:19 2022] PUT /identity/v3/projects/75434cc693884f639e8b84015d47cc58/users/0d4d9ce0bb284c378ab8627e107c20a5/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:05:19.659647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1f948-ca38-42c4-953f-3b37b549a17e None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:19.660153 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1f948-ca38-42c4-953f-3b37b549a17e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:19.660236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1f948-ca38-42c4-953f-3b37b549a17e None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:19.661541 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 132/254] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:19.667041 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4715d6-c24e-4fb8-86a5-8f8a59bb4932 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:19.667327 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4715d6-c24e-4fb8-86a5-8f8a59bb4932 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:19.667622 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4715d6-c24e-4fb8-86a5-8f8a59bb4932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:19.692731 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-2c4715d6-c24e-4fb8-86a5-8f8a59bb4932 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:19.694018 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-2c4715d6-c24e-4fb8-86a5-8f8a59bb4932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:19.758272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad0adf-a4f4-4df8-b8d6-9bd70404c9f8 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:19.758671 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad0adf-a4f4-4df8-b8d6-9bd70404c9f8 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:19.758923 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad0adf-a4f4-4df8-b8d6-9bd70404c9f8 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:19.760361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 133/255] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:19.769824 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6fa20-3d07-45ae-8810-ca0af3cb5dfe None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:19.770134 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6fa20-3d07-45ae-8810-ca0af3cb5dfe None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:19.770369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6fa20-3d07-45ae-8810-ca0af3cb5dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:19.806119 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-04f6fa20-3d07-45ae-8810-ca0af3cb5dfe None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:19.807487 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-04f6fa20-3d07-45ae-8810-ca0af3cb5dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:19.826362 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 123/256] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:19.891784 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:19.916480 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 134/257] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:20.209368 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4715d6-c24e-4fb8-86a5-8f8a59bb4932 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:20.217434 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6fa20-3d07-45ae-8810-ca0af3cb5dfe None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:20.276612 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:20.365713 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6fa20-3d07-45ae-8810-ca0af3cb5dfe None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:20.430541 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c37e3c-c0cb-4391-bb49-d214da890609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:20.431569 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c37e3c-c0cb-4391-bb49-d214da890609 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:20.432084 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c37e3c-c0cb-4391-bb49-d214da890609 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:20.432578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c37e3c-c0cb-4391-bb49-d214da890609 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:20.432682 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4715d6-c24e-4fb8-86a5-8f8a59bb4932 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:20.452455 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c37e3c-c0cb-4391-bb49-d214da890609 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:20.457578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-17c37e3c-c0cb-4391-bb49-d214da890609 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:05:20.457578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:20.457578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:20.457578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:20.457578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:20.457578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:20.457578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:20.457578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:20.457578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:20.457578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:20.457578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:20.457578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:20.457578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:20.457578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:20.457578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:20.457578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:20.458481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:20.458481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:20.458481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:20.458481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:20.458481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:20.458481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:20.458481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:20.458481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:20.458481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:20.458481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:20.458481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:20.458481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:20.458481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:20.458481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:20.458481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:20.458481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:20.459301 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:20.459301 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:20.459301 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:20.459301 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:20.459301 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:20.459301 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:20.459301 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:20.459301 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:20.459301 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:20.459301 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:20.459301 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:05:20.459301 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:20.462786 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 135/258] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Sep 5 12:05:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:05:20.472536 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c37e3c-c0cb-4391-bb49-d214da890609 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:20.541194 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b60ba5-08a9-45b7-9de7-d52cdb6dd356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:20.541853 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b60ba5-08a9-45b7-9de7-d52cdb6dd356 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:20.542137 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b60ba5-08a9-45b7-9de7-d52cdb6dd356 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:20.542415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b60ba5-08a9-45b7-9de7-d52cdb6dd356 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:20.543863 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:20.576481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b60ba5-08a9-45b7-9de7-d52cdb6dd356 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:20.585421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-84b60ba5-08a9-45b7-9de7-d52cdb6dd356 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:05:20.585421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:20.585421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:20.585421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:20.585421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:20.585421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:20.585421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:20.585421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:20.585421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:20.585421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:20.585421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:20.585421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:20.585421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:20.585421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:20.585421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:20.585421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:20.586367 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:20.586367 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:20.586367 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:20.586367 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:20.586367 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:20.586367 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:20.586367 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:20.586367 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:20.586367 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:20.586367 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:20.586367 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:20.586367 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:20.586367 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:20.586367 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:20.586367 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:20.586367 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:20.587244 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:20.587244 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:20.587244 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:20.587244 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:20.587244 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:20.587244 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:20.587244 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:20.587244 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:20.587244 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:20.587244 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:20.587244 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:05:20.587244 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:20.587244 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 124/259] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:05:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:20.592628 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b60ba5-08a9-45b7-9de7-d52cdb6dd356 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:20.635440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c37e3c-c0cb-4391-bb49-d214da890609 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:20.681465 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba204f1-53e3-44f6-a0d2-0808675308b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:20.682880 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba204f1-53e3-44f6-a0d2-0808675308b5 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:20.683349 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba204f1-53e3-44f6-a0d2-0808675308b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:20.684025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba204f1-53e3-44f6-a0d2-0808675308b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:20.685130 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba204f1-53e3-44f6-a0d2-0808675308b5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:20.685645 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba204f1-53e3-44f6-a0d2-0808675308b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:20.687930 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba204f1-53e3-44f6-a0d2-0808675308b5 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:20.698069 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 136/260] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:05:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:20.705723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba204f1-53e3-44f6-a0d2-0808675308b5 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:20.868921 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:20.893777 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b60ba5-08a9-45b7-9de7-d52cdb6dd356 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:20.901755 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba204f1-53e3-44f6-a0d2-0808675308b5 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:20.915921 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:20.954119 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d2b0d-44ba-4dc7-8997-f5684993ded1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:20.954845 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d2b0d-44ba-4dc7-8997-f5684993ded1 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:20.955128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d2b0d-44ba-4dc7-8997-f5684993ded1 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:20.955402 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d2b0d-44ba-4dc7-8997-f5684993ded1 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:20.956133 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3d2b0d-44ba-4dc7-8997-f5684993ded1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:20.956440 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3d2b0d-44ba-4dc7-8997-f5684993ded1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:20.958021 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3d2b0d-44ba-4dc7-8997-f5684993ded1 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:20.973918 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 125/261] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:20.976999 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b6cc1-d826-40f9-be69-8436404a17c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:20.977642 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b6cc1-d826-40f9-be69-8436404a17c8 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:20.977919 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b6cc1-d826-40f9-be69-8436404a17c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:20.978188 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b6cc1-d826-40f9-be69-8436404a17c8 None admin] PATH_INFO: `/v3/users/3b7c35d7330343e0a7a4458dbfa44989` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:20.980357 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b6cc1-d826-40f9-be69-8436404a17c8 None admin] RBAC: Authorizing `identity:get_user(user_id=3b7c35d7330343e0a7a4458dbfa44989)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:20.980588 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d2b0d-44ba-4dc7-8997-f5684993ded1 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:20.981804 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b6cc1-d826-40f9-be69-8436404a17c8 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:20.988322 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 137/262] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:05:20 2022] GET /identity/v3/users/3b7c35d7330343e0a7a4458dbfa44989 => generated 283 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:20.997879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b6cc1-d826-40f9-be69-8436404a17c8 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:21.163643 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b6cc1-d826-40f9-be69-8436404a17c8 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:21.216982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa474ebe-e90b-452e-9c7f-2838f6e93c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:21.218225 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa474ebe-e90b-452e-9c7f-2838f6e93c32 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:21.218477 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa474ebe-e90b-452e-9c7f-2838f6e93c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:21.218745 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa474ebe-e90b-452e-9c7f-2838f6e93c32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:21.229182 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d2b0d-44ba-4dc7-8997-f5684993ded1 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:21.313695 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a046a-bdb9-4e57-8fec-197c6f4db193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:21.314347 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a046a-bdb9-4e57-8fec-197c6f4db193 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:21.314632 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a046a-bdb9-4e57-8fec-197c6f4db193 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:21.314908 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a046a-bdb9-4e57-8fec-197c6f4db193 None admin] PATH_INFO: `/v3/users/3b7c35d7330343e0a7a4458dbfa44989` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:21.319112 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646a046a-bdb9-4e57-8fec-197c6f4db193 None admin] RBAC: Authorizing `identity:get_user(user_id=3b7c35d7330343e0a7a4458dbfa44989)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:21.320599 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646a046a-bdb9-4e57-8fec-197c6f4db193 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:21.322548 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 126/263] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:05:20 2022] GET /identity/v3/users/3b7c35d7330343e0a7a4458dbfa44989 => generated 283 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:21.328832 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a046a-bdb9-4e57-8fec-197c6f4db193 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:21.329515 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:21.373293 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa474ebe-e90b-452e-9c7f-2838f6e93c32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:21.373951 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa474ebe-e90b-452e-9c7f-2838f6e93c32 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:21.383891 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:21.508338 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a046a-bdb9-4e57-8fec-197c6f4db193 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:21.564347 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af8718-b156-4c74-b1dc-2b4f1ef1b91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:21.564975 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af8718-b156-4c74-b1dc-2b4f1ef1b91c None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:21.565261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af8718-b156-4c74-b1dc-2b4f1ef1b91c None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:21.565539 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af8718-b156-4c74-b1dc-2b4f1ef1b91c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:21.634318 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 138/264] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:05:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:05:21.644857 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa474ebe-e90b-452e-9c7f-2838f6e93c32 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:21.681390 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af8718-b156-4c74-b1dc-2b4f1ef1b91c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:21.682948 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af8718-b156-4c74-b1dc-2b4f1ef1b91c None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:21.728820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:21.842207 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:21.887831 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa474ebe-e90b-452e-9c7f-2838f6e93c32 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:21.894623 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 127/265] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:05:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:05:21.900139 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af8718-b156-4c74-b1dc-2b4f1ef1b91c None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:21.937516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec10184-e381-493a-883f-fcfab9ecdfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:21.938166 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec10184-e381-493a-883f-fcfab9ecdfb8 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:21.938448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec10184-e381-493a-883f-fcfab9ecdfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:21.938723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec10184-e381-493a-883f-fcfab9ecdfb8 None admin] PATH_INFO: `/v3/projects/75434cc693884f639e8b84015d47cc58` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:21.939895 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec10184-e381-493a-883f-fcfab9ecdfb8 None admin] RBAC: Authorizing `identity:get_project(project_id=75434cc693884f639e8b84015d47cc58)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:21.941578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec10184-e381-493a-883f-fcfab9ecdfb8 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:21.943552 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 139/266] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:21 2022] GET /identity/v3/projects/75434cc693884f639e8b84015d47cc58 => generated 302 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:21.945199 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c7fa6-aef3-4cd6-b180-c93a930cf4e7 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:21.945584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c7fa6-aef3-4cd6-b180-c93a930cf4e7 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:21.945960 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c7fa6-aef3-4cd6-b180-c93a930cf4e7 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:21.947564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 140/267] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:21.949486 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c7fa6-aef3-4cd6-b180-c93a930cf4e7 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:22.069730 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af8718-b156-4c74-b1dc-2b4f1ef1b91c None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:22.083211 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c7fa6-aef3-4cd6-b180-c93a930cf4e7 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:22.098767 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:22.109553 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368bbbe-741c-440c-a8ae-7e284d71bc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:22.110192 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368bbbe-741c-440c-a8ae-7e284d71bc61 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:22.110474 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368bbbe-741c-440c-a8ae-7e284d71bc61 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:22.110750 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368bbbe-741c-440c-a8ae-7e284d71bc61 None admin] PATH_INFO: `/v3/projects/75434cc693884f639e8b84015d47cc58` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:22.111930 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7368bbbe-741c-440c-a8ae-7e284d71bc61 None admin] RBAC: Authorizing `identity:get_project(project_id=75434cc693884f639e8b84015d47cc58)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:22.113377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7368bbbe-741c-440c-a8ae-7e284d71bc61 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:22.120025 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 128/268] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:21 2022] GET /identity/v3/projects/75434cc693884f639e8b84015d47cc58 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:22.121309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce25e7-44fa-4587-aadf-d666211c77f3 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:22.121598 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce25e7-44fa-4587-aadf-d666211c77f3 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:22.121881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce25e7-44fa-4587-aadf-d666211c77f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:22.132441 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788e76c-a034-4711-869c-6ba7ee877cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:22.133400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788e76c-a034-4711-869c-6ba7ee877cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:22.133875 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788e76c-a034-4711-869c-6ba7ee877cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:22.134349 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788e76c-a034-4711-869c-6ba7ee877cd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:22.135279 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788e76c-a034-4711-869c-6ba7ee877cd9 None admin] RBAC: Adding query filter params (role.id=20bf8c77837d43b590a2bd9a67d01f84, scope.project.id=75434cc693884f639e8b84015d47cc58, user.id=3b7c35d7330343e0a7a4458dbfa44989) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:22.135842 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788e76c-a034-4711-869c-6ba7ee877cd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:22.138355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788e76c-a034-4711-869c-6ba7ee877cd9 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:22.142663 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-fbce25e7-44fa-4587-aadf-d666211c77f3 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:22.143919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-fbce25e7-44fa-4587-aadf-d666211c77f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:22.148147 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 141/269] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:05:21 2022] GET /identity/v3/role_assignments?user.id=3b7c35d7330343e0a7a4458dbfa44989&scope.project.id=75434cc693884f639e8b84015d47cc58&role.id=20bf8c77837d43b590a2bd9a67d01f84 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:22.150409 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d62726-3414-439b-b8cc-ef6af415efb2 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:22.150690 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d62726-3414-439b-b8cc-ef6af415efb2 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:22.150962 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d62726-3414-439b-b8cc-ef6af415efb2 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:22.153203 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 142/270] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:22.157584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d62726-3414-439b-b8cc-ef6af415efb2 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:22.168744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:22.291392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 129/271] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:22.297685 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73742312-d857-4042-9af1-d9932fe91cb6 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:22.297685 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73742312-d857-4042-9af1-d9932fe91cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:22.297685 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73742312-d857-4042-9af1-d9932fe91cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:22.314611 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-73742312-d857-4042-9af1-d9932fe91cb6 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:22.316049 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-73742312-d857-4042-9af1-d9932fe91cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:22.384414 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d62726-3414-439b-b8cc-ef6af415efb2 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:22.438008 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 130/272] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:22.439885 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794cb6e-2986-46fa-8c50-851699c68b6e None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:22.440444 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794cb6e-2986-46fa-8c50-851699c68b6e None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:22.440971 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794cb6e-2986-46fa-8c50-851699c68b6e None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:22.442805 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 131/273] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:22.449929 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5708bbb-1808-4c8f-bb37-2521031e0bb4 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:22.450574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5708bbb-1808-4c8f-bb37-2521031e0bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:22.451159 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5708bbb-1808-4c8f-bb37-2521031e0bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:22.451723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b921aa-3027-4bbd-8a99-5c7b716e8b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:22.452386 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b921aa-3027-4bbd-8a99-5c7b716e8b8f None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:22.452714 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b921aa-3027-4bbd-8a99-5c7b716e8b8f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:22.453066 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b921aa-3027-4bbd-8a99-5c7b716e8b8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:22.454025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b921aa-3027-4bbd-8a99-5c7b716e8b8f None admin] RBAC: Adding query filter params (role.id=618c48706f7f445abeddf7eb2f8e927a, scope.project.id=75434cc693884f639e8b84015d47cc58, user.id=3b7c35d7330343e0a7a4458dbfa44989) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:22.454509 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b921aa-3027-4bbd-8a99-5c7b716e8b8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:22.459567 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b921aa-3027-4bbd-8a99-5c7b716e8b8f None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:22.464154 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 143/274] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:05:22 2022] GET /identity/v3/role_assignments?user.id=3b7c35d7330343e0a7a4458dbfa44989&scope.project.id=75434cc693884f639e8b84015d47cc58&role.id=618c48706f7f445abeddf7eb2f8e927a => generated 608 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:22.478880 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-e5708bbb-1808-4c8f-bb37-2521031e0bb4 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:22.480427 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-e5708bbb-1808-4c8f-bb37-2521031e0bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:22.545255 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:22.567167 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 132/275] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:05:22.614327 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b921aa-3027-4bbd-8a99-5c7b716e8b8f None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:22.708529 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e9453-8338-49b5-b01c-03874e5899b4 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:22.708836 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e9453-8338-49b5-b01c-03874e5899b4 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:22.709002 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e9453-8338-49b5-b01c-03874e5899b4 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:22.710386 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 133/276] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:05:22.716106 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74a032-d058-437f-9241-f3198c08caa8 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:22.716390 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74a032-d058-437f-9241-f3198c08caa8 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:22.716664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74a032-d058-437f-9241-f3198c08caa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:22.724323 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:22.737591 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-5c74a032-d058-437f-9241-f3198c08caa8 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:22.738804 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-5c74a032-d058-437f-9241-f3198c08caa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:22.785020 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b921aa-3027-4bbd-8a99-5c7b716e8b8f None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:22.823682 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 134/277] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:22.824629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74a032-d058-437f-9241-f3198c08caa8 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:22.837418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee05cd82-9719-43d1-b509-01d5b18b2bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:22.838056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05cd82-9719-43d1-b509-01d5b18b2bfe None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:22.838765 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05cd82-9719-43d1-b509-01d5b18b2bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:22.839044 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05cd82-9719-43d1-b509-01d5b18b2bfe None admin] PATH_INFO: `/v3/roles/member` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:22.853193 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee05cd82-9719-43d1-b509-01d5b18b2bfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:22.861164 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-ee05cd82-9719-43d1-b509-01d5b18b2bfe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:22.861164 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:22.861164 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:22.861164 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:22.861164 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:22.861164 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:22.861164 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:22.861164 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:22.861164 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:22.861164 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:22.861164 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:22.861164 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:22.861164 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:22.861164 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:22.861164 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:22.861164 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:22.862151 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:22.862151 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:22.862151 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:22.862151 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:22.862151 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:22.862151 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:22.862151 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:22.862151 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:22.862151 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:22.862151 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:22.862151 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:22.862151 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:22.862151 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:22.862151 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:22.862151 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:22.862151 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:22.862960 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:22.862960 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:22.862960 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:22.862960 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:22.862960 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:22.862960 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:22.862960 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:22.862960 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:22.862960 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:22.862960 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:22.862960 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:22.862960 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:22.862960 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 144/278] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:22.868734 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee05cd82-9719-43d1-b509-01d5b18b2bfe None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:22.945106 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74a032-d058-437f-9241-f3198c08caa8 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:22.962367 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:22.984774 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcea937-f4a7-47c7-9997-dae47c0b5ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:22.985407 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcea937-f4a7-47c7-9997-dae47c0b5ea0 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:22.985688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcea937-f4a7-47c7-9997-dae47c0b5ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:22.985961 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcea937-f4a7-47c7-9997-dae47c0b5ea0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:22.987286 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcea937-f4a7-47c7-9997-dae47c0b5ea0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:22.988785 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcea937-f4a7-47c7-9997-dae47c0b5ea0 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:22.989827 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee05cd82-9719-43d1-b509-01d5b18b2bfe None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:22.990542 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 135/279] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:05:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:22.995515 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcea937-f4a7-47c7-9997-dae47c0b5ea0 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:23.018964 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:23.025718 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb18adf2-e298-4ea2-9bb4-8d5436867f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:23.026439 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18adf2-e298-4ea2-9bb4-8d5436867f57 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:23.026613 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18adf2-e298-4ea2-9bb4-8d5436867f57 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:23.026879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18adf2-e298-4ea2-9bb4-8d5436867f57 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:23.027556 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb18adf2-e298-4ea2-9bb4-8d5436867f57 None admin] RBAC: Adding query filter params (name=member) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:23.027877 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb18adf2-e298-4ea2-9bb4-8d5436867f57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:23.029500 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb18adf2-e298-4ea2-9bb4-8d5436867f57 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:23.041649 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 145/280] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Sep 5 12:05:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:23.042723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb18adf2-e298-4ea2-9bb4-8d5436867f57 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:23.112609 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcea937-f4a7-47c7-9997-dae47c0b5ea0 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:23.145936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb18adf2-e298-4ea2-9bb4-8d5436867f57 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:23.146436 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e615d583-5432-40fe-8ebc-03cc302db194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:23.147056 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615d583-5432-40fe-8ebc-03cc302db194 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:23.147331 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615d583-5432-40fe-8ebc-03cc302db194 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:23.147693 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615d583-5432-40fe-8ebc-03cc302db194 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:23.158554 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e615d583-5432-40fe-8ebc-03cc302db194 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:23.163389 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-e615d583-5432-40fe-8ebc-03cc302db194 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:05:23.163389 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:23.163389 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:23.163389 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:23.163389 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:23.163389 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:23.163389 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:23.163389 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:23.163389 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:23.163389 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:23.163389 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:23.163389 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:23.163389 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:23.163389 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:23.163389 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:23.163389 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:23.164325 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:23.164325 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:23.164325 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:23.164325 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:23.164325 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:23.164325 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:23.164325 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:23.164325 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:23.164325 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:23.164325 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:23.164325 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:23.164325 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:23.164325 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:23.164325 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:23.164325 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:23.164325 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:23.165121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:23.165121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:23.165121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:23.165121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:23.165121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:23.165121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:23.165121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:23.165121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:23.165121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:23.165121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:23.165121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:05:23.165121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:23.165121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 136/281] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:23.165778 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e615d583-5432-40fe-8ebc-03cc302db194 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:23.181835 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78da40-d711-42ce-be31-a9423ea1a8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:23.182444 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78da40-d711-42ce-be31-a9423ea1a8f4 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:23.182717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78da40-d711-42ce-be31-a9423ea1a8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:23.182985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78da40-d711-42ce-be31-a9423ea1a8f4 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:23.183704 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f78da40-d711-42ce-be31-a9423ea1a8f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:23.185245 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f78da40-d711-42ce-be31-a9423ea1a8f4 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:23.205373 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 146/282] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Sep 5 12:05:23 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:05:23.206204 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78da40-d711-42ce-be31-a9423ea1a8f4 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:23.223083 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:23.277544 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:23.282086 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e615d583-5432-40fe-8ebc-03cc302db194 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:23.322222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535fecce-40dc-47bf-b978-36302635e3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:23.322872 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fecce-40dc-47bf-b978-36302635e3da None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:23.323133 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fecce-40dc-47bf-b978-36302635e3da None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:23.323395 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fecce-40dc-47bf-b978-36302635e3da None admin] PATH_INFO: `/v3/users/2d49d4470bbd4b5ebda6887318f8ee1c` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:23.338299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535fecce-40dc-47bf-b978-36302635e3da None admin] RBAC: Authorizing `identity:get_user(user_id=2d49d4470bbd4b5ebda6887318f8ee1c)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:23.339802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535fecce-40dc-47bf-b978-36302635e3da None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:23.341501 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 137/283] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:05:23 2022] GET /identity/v3/users/2d49d4470bbd4b5ebda6887318f8ee1c => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:23.342809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535fecce-40dc-47bf-b978-36302635e3da None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:23.350654 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78da40-d711-42ce-be31-a9423ea1a8f4 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:23.395480 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e4ae4-bba6-4baf-b3d1-52a4ad0086d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:23.396226 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e4ae4-bba6-4baf-b3d1-52a4ad0086d4 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:23.396500 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e4ae4-bba6-4baf-b3d1-52a4ad0086d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:23.396766 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e4ae4-bba6-4baf-b3d1-52a4ad0086d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:23.408890 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e4ae4-bba6-4baf-b3d1-52a4ad0086d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:23.415254 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-370e4ae4-bba6-4baf-b3d1-52a4ad0086d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:23.415254 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:23.415254 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:23.415254 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:23.415254 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:23.415254 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:23.415254 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:23.415254 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:23.415254 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:23.415254 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:23.415254 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:23.415254 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:23.415254 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:23.415254 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:23.415254 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:23.415254 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:23.416274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:23.416274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:23.416274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:23.416274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:23.416274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:23.416274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:23.416274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:23.416274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:23.416274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:23.416274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:23.416274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:23.416274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:23.416274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:23.416274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:23.416274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:23.416274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:23.417105 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:23.417105 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:23.417105 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:23.417105 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:23.417105 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:23.417105 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:23.417105 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:23.417105 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:23.417105 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:23.417105 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:23.417105 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:23.417105 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:23.417105 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 147/284] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:23.417105 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e4ae4-bba6-4baf-b3d1-52a4ad0086d4 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:23.482657 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535fecce-40dc-47bf-b978-36302635e3da None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:23.485575 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:23.531730 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc6a0a-6b36-48bc-a435-4aad56c487ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:23.532344 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc6a0a-6b36-48bc-a435-4aad56c487ca None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:23.532661 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc6a0a-6b36-48bc-a435-4aad56c487ca None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:23.532942 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc6a0a-6b36-48bc-a435-4aad56c487ca None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:23.533621 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc6a0a-6b36-48bc-a435-4aad56c487ca None admin] RBAC: Adding query filter params (name=reader) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:23.533932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc6a0a-6b36-48bc-a435-4aad56c487ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:23.535307 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc6a0a-6b36-48bc-a435-4aad56c487ca None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:23.544932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 138/285] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Sep 5 12:05:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:23.545702 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e4ae4-bba6-4baf-b3d1-52a4ad0086d4 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:23.545882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc6a0a-6b36-48bc-a435-4aad56c487ca None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:23.559757 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:23.587713 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b4693-9eda-43a4-bd09-e06c86278a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:23.588328 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b4693-9eda-43a4-bd09-e06c86278a7c None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:23.588610 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b4693-9eda-43a4-bd09-e06c86278a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:23.588883 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b4693-9eda-43a4-bd09-e06c86278a7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:23.589615 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b4693-9eda-43a4-bd09-e06c86278a7c None admin] RBAC: Adding query filter params (role.id=ede2986592354e9888dde3123b2d8c82, user.id=2d49d4470bbd4b5ebda6887318f8ee1c, scope.system=all) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:23.589926 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b4693-9eda-43a4-bd09-e06c86278a7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:23.591652 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b4693-9eda-43a4-bd09-e06c86278a7c None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:23.597815 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 148/286] 10.222.0.16 () {56 vars in 1308 bytes} [Mon Sep 5 12:05:23 2022] GET /identity/v3/role_assignments?user.id=2d49d4470bbd4b5ebda6887318f8ee1c&scope.system=all&role.id=ede2986592354e9888dde3123b2d8c82 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:23.598520 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b4693-9eda-43a4-bd09-e06c86278a7c None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:23.693054 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc6a0a-6b36-48bc-a435-4aad56c487ca None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:23.739081 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0562e0d9-a35d-49b1-8c17-b0c9720ca6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:23.739652 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562e0d9-a35d-49b1-8c17-b0c9720ca6cd None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:23.741113 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562e0d9-a35d-49b1-8c17-b0c9720ca6cd None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:23.742449 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562e0d9-a35d-49b1-8c17-b0c9720ca6cd None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:23.743184 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0562e0d9-a35d-49b1-8c17-b0c9720ca6cd None admin] RBAC: Adding query filter params (name=member) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:23.743490 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0562e0d9-a35d-49b1-8c17-b0c9720ca6cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:23.744928 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0562e0d9-a35d-49b1-8c17-b0c9720ca6cd None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:23.759247 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 139/287] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Sep 5 12:05:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:23.765684 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0562e0d9-a35d-49b1-8c17-b0c9720ca6cd None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:23.794069 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b4693-9eda-43a4-bd09-e06c86278a7c None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:23.801477 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:23.836548 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2bc5d6-54ee-4c69-9240-ebf33933a4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:23.837318 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bc5d6-54ee-4c69-9240-ebf33933a4f0 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:23.837670 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bc5d6-54ee-4c69-9240-ebf33933a4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:23.838006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bc5d6-54ee-4c69-9240-ebf33933a4f0 None admin] PATH_INFO: `/v3/users/e31a11cf45f14b89bc3c55c95dc76009` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:23.839977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2bc5d6-54ee-4c69-9240-ebf33933a4f0 None admin] RBAC: Authorizing `identity:get_user(user_id=e31a11cf45f14b89bc3c55c95dc76009)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:23.841491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2bc5d6-54ee-4c69-9240-ebf33933a4f0 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:23.843498 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 149/288] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:05:23 2022] GET /identity/v3/users/e31a11cf45f14b89bc3c55c95dc76009 => generated 297 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:23.849652 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2bc5d6-54ee-4c69-9240-ebf33933a4f0 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:23.918773 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:23.948547 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0562e0d9-a35d-49b1-8c17-b0c9720ca6cd None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:23.985264 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ffc20-6cfb-4080-a2a1-a666fcc2fdb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:23.986038 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ffc20-6cfb-4080-a2a1-a666fcc2fdb0 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:23.986410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ffc20-6cfb-4080-a2a1-a666fcc2fdb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:23.986757 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ffc20-6cfb-4080-a2a1-a666fcc2fdb0 None admin] PATH_INFO: `/v3/users/0d4d9ce0bb284c378ab8627e107c20a5` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:23.988065 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ffc20-6cfb-4080-a2a1-a666fcc2fdb0 None admin] RBAC: Authorizing `identity:get_user(user_id=0d4d9ce0bb284c378ab8627e107c20a5)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:23.989756 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ffc20-6cfb-4080-a2a1-a666fcc2fdb0 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:23.991645 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 140/289] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:05:23 2022] GET /identity/v3/users/0d4d9ce0bb284c378ab8627e107c20a5 => generated 297 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:23.997359 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ffc20-6cfb-4080-a2a1-a666fcc2fdb0 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:24.047028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2bc5d6-54ee-4c69-9240-ebf33933a4f0 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:24.085709 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d801182-293d-4452-9c9b-9fd6c7290be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:24.086344 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d801182-293d-4452-9c9b-9fd6c7290be3 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:24.086621 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d801182-293d-4452-9c9b-9fd6c7290be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:24.086893 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d801182-293d-4452-9c9b-9fd6c7290be3 None admin] PATH_INFO: `/v3/projects/75434cc693884f639e8b84015d47cc58` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:24.088953 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d801182-293d-4452-9c9b-9fd6c7290be3 None admin] RBAC: Authorizing `identity:get_project(project_id=75434cc693884f639e8b84015d47cc58)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:24.090530 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d801182-293d-4452-9c9b-9fd6c7290be3 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:24.092251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 150/290] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:23 2022] GET /identity/v3/projects/75434cc693884f639e8b84015d47cc58 => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:24.098853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d801182-293d-4452-9c9b-9fd6c7290be3 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:24.218336 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d801182-293d-4452-9c9b-9fd6c7290be3 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:24.228516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:24.236668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:24.264687 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ffc20-6cfb-4080-a2a1-a666fcc2fdb0 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:24.267359 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f3f09-e1f3-4e35-8b59-f8b71f6b1063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:24.268286 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f3f09-e1f3-4e35-8b59-f8b71f6b1063 None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:24.268709 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f3f09-e1f3-4e35-8b59-f8b71f6b1063 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:24.269150 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f3f09-e1f3-4e35-8b59-f8b71f6b1063 None admin] PATH_INFO: `/v3/projects/75434cc693884f639e8b84015d47cc58/users/e31a11cf45f14b89bc3c55c95dc76009/roles/290929f17e764f019e358e7751365fd5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:24.272097 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679f3f09-e1f3-4e35-8b59-f8b71f6b1063 None admin] RBAC: Authorizing `identity:create_grant(project_id=75434cc693884f639e8b84015d47cc58, user_id=e31a11cf45f14b89bc3c55c95dc76009, role_id=290929f17e764f019e358e7751365fd5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:24.273825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679f3f09-e1f3-4e35-8b59-f8b71f6b1063 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:24.298207 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 151/291] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Sep 5 12:05:24 2022] PUT /identity/v3/projects/75434cc693884f639e8b84015d47cc58/users/e31a11cf45f14b89bc3c55c95dc76009/roles/290929f17e764f019e358e7751365fd5 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:05:24.324827 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71efd4a4-6ed7-4ad2-91a1-a701e67435ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:24.328915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efd4a4-6ed7-4ad2-91a1-a701e67435ac None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:24.329385 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efd4a4-6ed7-4ad2-91a1-a701e67435ac None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:24.329805 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efd4a4-6ed7-4ad2-91a1-a701e67435ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:24.437810 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71efd4a4-6ed7-4ad2-91a1-a701e67435ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:24.439317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71efd4a4-6ed7-4ad2-91a1-a701e67435ac None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:24.558755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:24.570370 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 141/292] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:05:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:05:24.575845 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f3f09-e1f3-4e35-8b59-f8b71f6b1063 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:24.691180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f3f09-e1f3-4e35-8b59-f8b71f6b1063 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:24.726822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e12ca9-e19e-4066-98c1-834559b736a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:24.727453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e12ca9-e19e-4066-98c1-834559b736a6 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:24.727765 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e12ca9-e19e-4066-98c1-834559b736a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:24.728043 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e12ca9-e19e-4066-98c1-834559b736a6 None admin] PATH_INFO: `/v3/projects/75434cc693884f639e8b84015d47cc58` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:24.729178 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e12ca9-e19e-4066-98c1-834559b736a6 None admin] RBAC: Authorizing `identity:get_project(project_id=75434cc693884f639e8b84015d47cc58)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:24.730618 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e12ca9-e19e-4066-98c1-834559b736a6 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:24.733422 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 152/293] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:24 2022] GET /identity/v3/projects/75434cc693884f639e8b84015d47cc58 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:24.739165 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71efd4a4-6ed7-4ad2-91a1-a701e67435ac None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:24.889740 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71efd4a4-6ed7-4ad2-91a1-a701e67435ac None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:24.948090 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94639b-29fb-4c64-b861-db8689ba86f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:24.949117 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94639b-29fb-4c64-b861-db8689ba86f2 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:24.949620 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94639b-29fb-4c64-b861-db8689ba86f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:24.950117 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94639b-29fb-4c64-b861-db8689ba86f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:24.951408 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff94639b-29fb-4c64-b861-db8689ba86f2 None admin] RBAC: Adding query filter params (role.id=ede2986592354e9888dde3123b2d8c82, scope.project.id=75434cc693884f639e8b84015d47cc58, user.id=0d4d9ce0bb284c378ab8627e107c20a5) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:24.952554 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff94639b-29fb-4c64-b861-db8689ba86f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:24.955390 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff94639b-29fb-4c64-b861-db8689ba86f2 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:24.966626 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 142/294] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:05:24 2022] GET /identity/v3/role_assignments?user.id=0d4d9ce0bb284c378ab8627e107c20a5&scope.project.id=75434cc693884f639e8b84015d47cc58&role.id=ede2986592354e9888dde3123b2d8c82 => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:25.539353 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe4369-1e7d-47c3-a68f-2c0bff5c6806 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:25.539877 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe4369-1e7d-47c3-a68f-2c0bff5c6806 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:25.540245 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe4369-1e7d-47c3-a68f-2c0bff5c6806 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:25.541601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 153/295] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:25.547189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec971f4-c686-44e5-a745-553e8752a45a None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:25.547691 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec971f4-c686-44e5-a745-553e8752a45a None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:25.547883 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec971f4-c686-44e5-a745-553e8752a45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:25.570619 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-aec971f4-c686-44e5-a745-553e8752a45a None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:25.571849 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-aec971f4-c686-44e5-a745-553e8752a45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:25.642898 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:25.663714 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 143/296] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:25.738627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eddd71-a8de-43f8-a7f8-835212df9d61 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:25.738910 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eddd71-a8de-43f8-a7f8-835212df9d61 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:25.739189 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eddd71-a8de-43f8-a7f8-835212df9d61 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:25.741032 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 154/297] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:25.746817 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85924f0-4b83-4fcb-93aa-b51bc400b634 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:25.747125 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85924f0-4b83-4fcb-93aa-b51bc400b634 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:25.747387 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85924f0-4b83-4fcb-93aa-b51bc400b634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:25.766675 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-a85924f0-4b83-4fcb-93aa-b51bc400b634 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:25.767928 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-a85924f0-4b83-4fcb-93aa-b51bc400b634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:25.843076 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 144/298] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:25.946641 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eddd71-a8de-43f8-a7f8-835212df9d61 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:25.950655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fb07b-05b8-4425-8e34-277765f060cb None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:25.950944 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fb07b-05b8-4425-8e34-277765f060cb None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:25.951286 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fb07b-05b8-4425-8e34-277765f060cb None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:25.952837 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 145/299] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:05:25.957575 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb3b84-2618-49e4-9232-5a5c7825d496 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:25.957864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb3b84-2618-49e4-9232-5a5c7825d496 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:25.958134 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb3b84-2618-49e4-9232-5a5c7825d496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:25.977012 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-1ecb3b84-2618-49e4-9232-5a5c7825d496 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:25.978482 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-1ecb3b84-2618-49e4-9232-5a5c7825d496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:25.989676 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:26.067725 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eddd71-a8de-43f8-a7f8-835212df9d61 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:26.067940 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 146/300] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:26.102230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fff28-7506-4176-a503-64a468f47bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:26.102953 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fff28-7506-4176-a503-64a468f47bd3 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:26.103234 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fff28-7506-4176-a503-64a468f47bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:26.103502 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fff28-7506-4176-a503-64a468f47bd3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:26.114900 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42fff28-7506-4176-a503-64a468f47bd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:26.119725 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-c42fff28-7506-4176-a503-64a468f47bd3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:05:26.119725 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:26.119725 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:26.119725 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:26.119725 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:26.119725 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:26.119725 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:26.119725 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:26.119725 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:26.119725 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:26.119725 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:26.119725 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:26.119725 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:26.119725 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:26.119725 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:26.119725 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:26.120630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:26.120630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:26.120630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:26.120630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:26.120630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:26.120630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:26.120630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:26.120630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:26.120630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:26.120630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:26.120630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:26.120630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:26.120630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:26.120630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:26.120630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:26.120630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:26.121837 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:26.121837 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:26.121837 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:26.121837 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:26.121837 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:26.121837 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:26.121837 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:26.121837 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:26.121837 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:26.121837 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:26.121837 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:05:26.121837 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:26.121837 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 155/301] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:26.126873 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb3b84-2618-49e4-9232-5a5c7825d496 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:26.136424 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fff28-7506-4176-a503-64a468f47bd3 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:26.181238 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:26.244392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb3b84-2618-49e4-9232-5a5c7825d496 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:26.247857 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:26.254964 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fff28-7506-4176-a503-64a468f47bd3 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:26.294629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3982bb-a3f0-4e8f-8fef-a1e1adf8a8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:26.296446 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3982bb-a3f0-4e8f-8fef-a1e1adf8a8c5 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:26.297030 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3982bb-a3f0-4e8f-8fef-a1e1adf8a8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:26.297573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3982bb-a3f0-4e8f-8fef-a1e1adf8a8c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:26.298490 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3982bb-a3f0-4e8f-8fef-a1e1adf8a8c5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:26.299117 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3982bb-a3f0-4e8f-8fef-a1e1adf8a8c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:26.299827 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c1f34-ccc0-423b-a03c-b0032a12d0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:26.300571 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c1f34-ccc0-423b-a03c-b0032a12d0d9 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:26.300910 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c1f34-ccc0-423b-a03c-b0032a12d0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:26.301263 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c1f34-ccc0-423b-a03c-b0032a12d0d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:26.301760 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3982bb-a3f0-4e8f-8fef-a1e1adf8a8c5 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:26.302584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c1f34-ccc0-423b-a03c-b0032a12d0d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:26.304164 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c1f34-ccc0-423b-a03c-b0032a12d0d9 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:26.310533 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 156/302] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:05:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:26.311565 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c1f34-ccc0-423b-a03c-b0032a12d0d9 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:26.317382 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 147/303] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Sep 5 12:05:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:26.327367 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3982bb-a3f0-4e8f-8fef-a1e1adf8a8c5 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:26.437230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c1f34-ccc0-423b-a03c-b0032a12d0d9 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:26.445865 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3982bb-a3f0-4e8f-8fef-a1e1adf8a8c5 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:26.456355 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:26.474630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9a5b3-5720-4705-a99c-a120b2254a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:26.475245 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9a5b3-5720-4705-a99c-a120b2254a8a None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:26.475554 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9a5b3-5720-4705-a99c-a120b2254a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:26.475825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9a5b3-5720-4705-a99c-a120b2254a8a None admin] PATH_INFO: `/v3/groups` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:26.476460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf9a5b3-5720-4705-a99c-a120b2254a8a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:26.477893 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf9a5b3-5720-4705-a99c-a120b2254a8a None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:26.486618 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282907d-94b3-422b-8f8e-361792a34b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:26.487392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282907d-94b3-422b-8f8e-361792a34b97 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:26.487735 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282907d-94b3-422b-8f8e-361792a34b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:26.488154 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282907d-94b3-422b-8f8e-361792a34b97 None admin] PATH_INFO: `/v3/users/37371f05b69844ecbee81b54b62a5a39` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:26.497361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 157/304] 10.222.0.16 () {60 vars in 1134 bytes} [Mon Sep 5 12:05:26 2022] POST /identity/v3/groups => generated 229 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:05:26.498450 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9a5b3-5720-4705-a99c-a120b2254a8a None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:26.517471 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d282907d-94b3-422b-8f8e-361792a34b97 None admin] RBAC: Authorizing `identity:get_user(user_id=37371f05b69844ecbee81b54b62a5a39)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:26.520040 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d282907d-94b3-422b-8f8e-361792a34b97 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:26.527102 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 148/305] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:05:26 2022] GET /identity/v3/users/37371f05b69844ecbee81b54b62a5a39 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:26.529290 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282907d-94b3-422b-8f8e-361792a34b97 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:26.532388 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:26.627726 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9a5b3-5720-4705-a99c-a120b2254a8a None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:26.666952 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df54d5-a766-496e-b3ea-b6c9cbea28d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:26.668659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df54d5-a766-496e-b3ea-b6c9cbea28d2 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:26.668659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df54d5-a766-496e-b3ea-b6c9cbea28d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:26.668659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df54d5-a766-496e-b3ea-b6c9cbea28d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:26.679490 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282907d-94b3-422b-8f8e-361792a34b97 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:26.680158 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df54d5-a766-496e-b3ea-b6c9cbea28d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:26.685614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-10df54d5-a766-496e-b3ea-b6c9cbea28d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:26.685614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:26.685614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:26.685614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:26.685614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:26.685614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:26.685614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:26.685614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:26.685614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:26.685614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:26.685614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:26.685614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:26.685614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:26.685614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:26.685614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:26.685614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:26.686584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:26.686584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:26.686584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:26.686584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:26.686584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:26.686584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:26.686584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:26.686584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:26.686584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:26.686584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:26.686584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:26.686584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:26.686584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:26.686584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:26.686584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:26.686584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:26.692475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:26.692475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:26.692475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:26.692475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:26.692475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:26.692475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:26.692475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:26.692475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:26.692475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:26.692475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:26.692475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:26.692475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:26.692475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 158/306] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:26.692475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf96b7d-1c27-4ab8-9fab-1a5c12e9b067 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:26.693403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf96b7d-1c27-4ab8-9fab-1a5c12e9b067 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:26.693403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf96b7d-1c27-4ab8-9fab-1a5c12e9b067 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:26.693403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 159/307] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:26.694497 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf96b7d-1c27-4ab8-9fab-1a5c12e9b067 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:26.720983 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0654a35a-4668-4702-b8f8-e724961eac60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:26.721601 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654a35a-4668-4702-b8f8-e724961eac60 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:26.721884 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654a35a-4668-4702-b8f8-e724961eac60 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:26.722159 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654a35a-4668-4702-b8f8-e724961eac60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:26.722895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0654a35a-4668-4702-b8f8-e724961eac60 None admin] RBAC: Adding query filter params (role.id=290929f17e764f019e358e7751365fd5, user.id=37371f05b69844ecbee81b54b62a5a39, scope.system=all) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:26.723208 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0654a35a-4668-4702-b8f8-e724961eac60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:26.724684 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0654a35a-4668-4702-b8f8-e724961eac60 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:26.735563 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 149/308] 10.222.0.16 () {56 vars in 1308 bytes} [Mon Sep 5 12:05:26 2022] GET /identity/v3/role_assignments?user.id=37371f05b69844ecbee81b54b62a5a39&scope.system=all&role.id=290929f17e764f019e358e7751365fd5 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:26.735563 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a335497-1ca8-449f-917c-03afd507ddc1 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:26.735563 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a335497-1ca8-449f-917c-03afd507ddc1 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:26.735563 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a335497-1ca8-449f-917c-03afd507ddc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:26.754591 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-1a335497-1ca8-449f-917c-03afd507ddc1 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:26.756029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-1a335497-1ca8-449f-917c-03afd507ddc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:26.790396 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:26.817787 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:26.844785 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf96b7d-1c27-4ab8-9fab-1a5c12e9b067 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:26.846884 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 150/309] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:26.884475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c41d67-0413-4af3-9c89-4c294f204d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:26.885355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c41d67-0413-4af3-9c89-4c294f204d4f None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:26.885784 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c41d67-0413-4af3-9c89-4c294f204d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:26.886230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c41d67-0413-4af3-9c89-4c294f204d4f None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:26.887065 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c41d67-0413-4af3-9c89-4c294f204d4f None admin] RBAC: Adding query filter params (name=member) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:26.887540 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c41d67-0413-4af3-9c89-4c294f204d4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:26.889690 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c41d67-0413-4af3-9c89-4c294f204d4f None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:26.899051 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 160/310] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Sep 5 12:05:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:26.905959 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a335497-1ca8-449f-917c-03afd507ddc1 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:27.014954 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a335497-1ca8-449f-917c-03afd507ddc1 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:27.052085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c79225-34c4-4718-922b-c1a3dd346d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:27.052770 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c79225-34c4-4718-922b-c1a3dd346d87 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:27.053057 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c79225-34c4-4718-922b-c1a3dd346d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:27.053330 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c79225-34c4-4718-922b-c1a3dd346d87 None admin] PATH_INFO: `/v3/users/2d49d4470bbd4b5ebda6887318f8ee1c` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:27.054717 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c79225-34c4-4718-922b-c1a3dd346d87 None admin] RBAC: Authorizing `identity:get_user(user_id=2d49d4470bbd4b5ebda6887318f8ee1c)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:27.056251 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c79225-34c4-4718-922b-c1a3dd346d87 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:27.063097 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 151/311] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:05:26 2022] GET /identity/v3/users/2d49d4470bbd4b5ebda6887318f8ee1c => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:27.064638 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c41d67-0413-4af3-9c89-4c294f204d4f None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:27.122837 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c79225-34c4-4718-922b-c1a3dd346d87 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:27.159835 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:27.181206 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c41d67-0413-4af3-9c89-4c294f204d4f None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:27.219779 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59e861-6bca-411b-bd9a-ec81a794a3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:27.220448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59e861-6bca-411b-bd9a-ec81a794a3f2 None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:27.220730 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59e861-6bca-411b-bd9a-ec81a794a3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:27.221106 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59e861-6bca-411b-bd9a-ec81a794a3f2 None admin] PATH_INFO: `/v3/system/users/2d49d4470bbd4b5ebda6887318f8ee1c/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:27.222576 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be59e861-6bca-411b-bd9a-ec81a794a3f2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2d49d4470bbd4b5ebda6887318f8ee1c, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:27.224114 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be59e861-6bca-411b-bd9a-ec81a794a3f2 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:27.235913 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 161/312] 10.222.0.16 () {58 vars in 1334 bytes} [Mon Sep 5 12:05:27 2022] PUT /identity/v3/system/users/2d49d4470bbd4b5ebda6887318f8ee1c/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:05:27.269788 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c79225-34c4-4718-922b-c1a3dd346d87 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:27.314060 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f18f743-a330-46cd-a51b-d6517aae9f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:27.314782 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18f743-a330-46cd-a51b-d6517aae9f25 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:27.315062 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18f743-a330-46cd-a51b-d6517aae9f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:27.315335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18f743-a330-46cd-a51b-d6517aae9f25 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:27.328150 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f18f743-a330-46cd-a51b-d6517aae9f25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:27.333459 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-9f18f743-a330-46cd-a51b-d6517aae9f25 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:05:27.333459 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:27.333459 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:27.333459 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:27.333459 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:27.333459 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:27.333459 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:27.333459 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:27.333459 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:27.333459 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:27.333459 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:27.333459 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:27.333459 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:27.333459 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:27.333459 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:27.333459 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:27.334424 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:27.334424 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:27.334424 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:27.334424 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:27.334424 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:27.334424 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:27.334424 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:27.334424 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:27.334424 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:27.334424 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:27.334424 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:27.334424 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:27.334424 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:27.334424 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:27.334424 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:27.334424 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:27.335255 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:27.335255 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:27.335255 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:27.335255 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:27.335255 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:27.335255 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:27.335255 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:27.335255 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:27.335255 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:27.335255 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:27.335255 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:05:27.335255 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:27.335255 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 152/313] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:05:27.340722 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59e861-6bca-411b-bd9a-ec81a794a3f2 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:27.357109 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:27.460944 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59e861-6bca-411b-bd9a-ec81a794a3f2 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:27.499124 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d6619c-bb43-4047-8429-222f1af4a876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:27.500048 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6619c-bb43-4047-8429-222f1af4a876 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:27.500469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6619c-bb43-4047-8429-222f1af4a876 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:27.500910 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6619c-bb43-4047-8429-222f1af4a876 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:27.501712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d6619c-bb43-4047-8429-222f1af4a876 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:27.502156 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d6619c-bb43-4047-8429-222f1af4a876 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:27.503691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d6619c-bb43-4047-8429-222f1af4a876 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:27.512781 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 162/314] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Sep 5 12:05:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:27.518835 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f18f743-a330-46cd-a51b-d6517aae9f25 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:27.615141 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:27.628827 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f18f743-a330-46cd-a51b-d6517aae9f25 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:27.670288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49aa5c4-f106-4494-b75b-b22026e2516c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:27.671706 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49aa5c4-f106-4494-b75b-b22026e2516c None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:27.672004 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49aa5c4-f106-4494-b75b-b22026e2516c None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:27.672281 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49aa5c4-f106-4494-b75b-b22026e2516c None admin] PATH_INFO: `/v3/users/e31a11cf45f14b89bc3c55c95dc76009` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:27.673620 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49aa5c4-f106-4494-b75b-b22026e2516c None admin] RBAC: Authorizing `identity:get_user(user_id=e31a11cf45f14b89bc3c55c95dc76009)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:27.675398 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49aa5c4-f106-4494-b75b-b22026e2516c None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:27.677768 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 153/315] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:05:27 2022] GET /identity/v3/users/e31a11cf45f14b89bc3c55c95dc76009 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:27.684998 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d6619c-bb43-4047-8429-222f1af4a876 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:27.765453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:27.800695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d6619c-bb43-4047-8429-222f1af4a876 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:27.839565 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef20460-bd46-462d-b302-6c55d39f8b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:27.840564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef20460-bd46-462d-b302-6c55d39f8b91 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:27.841056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef20460-bd46-462d-b302-6c55d39f8b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:27.841535 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef20460-bd46-462d-b302-6c55d39f8b91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:27.916911 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef20460-bd46-462d-b302-6c55d39f8b91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:27.918785 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef20460-bd46-462d-b302-6c55d39f8b91 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:27.991777 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:28.024730 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 163/316] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:05:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:05:28.031453 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49aa5c4-f106-4494-b75b-b22026e2516c None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:28.140516 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49aa5c4-f106-4494-b75b-b22026e2516c None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:28.177292 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9575e93-6900-4af2-990e-94e3738a4203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:28.177848 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9575e93-6900-4af2-990e-94e3738a4203 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:28.178126 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9575e93-6900-4af2-990e-94e3738a4203 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:28.178399 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9575e93-6900-4af2-990e-94e3738a4203 None admin] PATH_INFO: `/v3/projects/75434cc693884f639e8b84015d47cc58` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:28.179644 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9575e93-6900-4af2-990e-94e3738a4203 None admin] RBAC: Authorizing `identity:get_project(project_id=75434cc693884f639e8b84015d47cc58)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:28.181143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9575e93-6900-4af2-990e-94e3738a4203 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:28.182874 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 154/317] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:28 2022] GET /identity/v3/projects/75434cc693884f639e8b84015d47cc58 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:28.188971 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef20460-bd46-462d-b302-6c55d39f8b91 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:28.296927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef20460-bd46-462d-b302-6c55d39f8b91 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:28.333420 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70411586-658c-49ce-bb2e-afa7b6897299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:28.334374 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70411586-658c-49ce-bb2e-afa7b6897299 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:28.334852 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70411586-658c-49ce-bb2e-afa7b6897299 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:28.335337 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70411586-658c-49ce-bb2e-afa7b6897299 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:28.336334 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70411586-658c-49ce-bb2e-afa7b6897299 None admin] RBAC: Adding query filter params (role.id=290929f17e764f019e358e7751365fd5, scope.project.id=75434cc693884f639e8b84015d47cc58, user.id=e31a11cf45f14b89bc3c55c95dc76009) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:28.336846 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70411586-658c-49ce-bb2e-afa7b6897299 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:28.338498 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70411586-658c-49ce-bb2e-afa7b6897299 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:28.345977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 164/318] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:05:28 2022] GET /identity/v3/role_assignments?user.id=e31a11cf45f14b89bc3c55c95dc76009&scope.project.id=75434cc693884f639e8b84015d47cc58&role.id=290929f17e764f019e358e7751365fd5 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:28.771786 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb89159-5c31-40da-8c7d-33d714ad61c0 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:28.772163 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb89159-5c31-40da-8c7d-33d714ad61c0 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:28.772344 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb89159-5c31-40da-8c7d-33d714ad61c0 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:28.773729 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 155/319] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:28.779309 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16101f0f-c324-4f1a-bfde-8cff4b54f967 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:28.779846 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16101f0f-c324-4f1a-bfde-8cff4b54f967 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:28.780019 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16101f0f-c324-4f1a-bfde-8cff4b54f967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:28.801371 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-16101f0f-c324-4f1a-bfde-8cff4b54f967 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:28.802573 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-16101f0f-c324-4f1a-bfde-8cff4b54f967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:28.830519 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434dbb77-1ee7-4240-b20c-ee28ac4f89ab None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:28.830859 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434dbb77-1ee7-4240-b20c-ee28ac4f89ab None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:28.831068 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434dbb77-1ee7-4240-b20c-ee28ac4f89ab None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:28.832450 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 156/320] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:05:28.837239 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12441c12-4af2-4345-98b1-51c1ac80ea39 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:28.837519 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12441c12-4af2-4345-98b1-51c1ac80ea39 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:28.837789 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12441c12-4af2-4345-98b1-51c1ac80ea39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:28.849260 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:28.858440 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-12441c12-4af2-4345-98b1-51c1ac80ea39 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:28.859694 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-12441c12-4af2-4345-98b1-51c1ac80ea39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:28.879127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:28.894192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 165/321] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:28.935683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 157/322] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:05:29.168300 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16101f0f-c324-4f1a-bfde-8cff4b54f967 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:29.203233 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12441c12-4af2-4345-98b1-51c1ac80ea39 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:29.280912 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16101f0f-c324-4f1a-bfde-8cff4b54f967 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:29.308225 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12441c12-4af2-4345-98b1-51c1ac80ea39 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:29.316569 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d55f285-63b9-4f04-bac4-223a7f4ca659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:29.317198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55f285-63b9-4f04-bac4-223a7f4ca659 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:29.317477 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55f285-63b9-4f04-bac4-223a7f4ca659 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:29.317749 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55f285-63b9-4f04-bac4-223a7f4ca659 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:29.328625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d55f285-63b9-4f04-bac4-223a7f4ca659 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:29.333448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-3d55f285-63b9-4f04-bac4-223a7f4ca659 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:05:29.333448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:29.333448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:29.333448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:29.333448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:29.333448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:29.333448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:29.333448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:29.333448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:29.333448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:29.333448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:29.333448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:29.333448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:29.333448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:29.333448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:29.333448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:29.334361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:29.334361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:29.334361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:29.334361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:29.334361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:29.334361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:29.334361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:29.334361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:29.334361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:29.334361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:29.334361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:29.334361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:29.334361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:29.334361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:29.334361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:29.334361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:29.335192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:29.335192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:29.335192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:29.335192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:29.335192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:29.335192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:29.335192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:29.335192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:29.335192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:29.335192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:29.335192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:05:29.335192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:29.335192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 166/323] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:29.336315 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55689906-f317-407c-a3df-0d16c8751b7e None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:29.336597 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55689906-f317-407c-a3df-0d16c8751b7e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:29.336866 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55689906-f317-407c-a3df-0d16c8751b7e None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:29.338250 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 167/324] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:05:29.340388 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55689906-f317-407c-a3df-0d16c8751b7e None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:29.347482 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a1be2-1138-481b-b6c8-08b60eb61e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:29.348150 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a1be2-1138-481b-b6c8-08b60eb61e31 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:29.348431 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a1be2-1138-481b-b6c8-08b60eb61e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:29.348710 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a1be2-1138-481b-b6c8-08b60eb61e31 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:29.349817 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8a1be2-1138-481b-b6c8-08b60eb61e31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:29.351256 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8a1be2-1138-481b-b6c8-08b60eb61e31 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:29.352999 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 158/325] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:05:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:29.353966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b43baa-cc5a-4513-8735-f0bdec684206 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:29.354229 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b43baa-cc5a-4513-8735-f0bdec684206 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:29.354501 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b43baa-cc5a-4513-8735-f0bdec684206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:29.372513 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:29.375393 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-77b43baa-cc5a-4513-8735-f0bdec684206 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:29.376636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-77b43baa-cc5a-4513-8735-f0bdec684206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:29.396224 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:29.447946 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 159/326] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:29.448788 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b43baa-cc5a-4513-8735-f0bdec684206 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:29.449672 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55689906-f317-407c-a3df-0d16c8751b7e None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:29.487405 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55389a8a-6014-4da6-ab48-265e93786971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:29.488051 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55389a8a-6014-4da6-ab48-265e93786971 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:29.488331 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55389a8a-6014-4da6-ab48-265e93786971 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:29.488717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55389a8a-6014-4da6-ab48-265e93786971 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:29.489501 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55389a8a-6014-4da6-ab48-265e93786971 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:29.489877 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55389a8a-6014-4da6-ab48-265e93786971 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:29.491499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55389a8a-6014-4da6-ab48-265e93786971 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:29.505647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 168/327] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Sep 5 12:05:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:29.506766 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55389a8a-6014-4da6-ab48-265e93786971 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:29.569787 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b43baa-cc5a-4513-8735-f0bdec684206 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:29.596789 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:29.609135 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c043ed-529e-4c96-990d-26d5e9a46606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:29.609361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55389a8a-6014-4da6-ab48-265e93786971 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:29.609811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c043ed-529e-4c96-990d-26d5e9a46606 None admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:29.610094 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c043ed-529e-4c96-990d-26d5e9a46606 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:29.610369 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c043ed-529e-4c96-990d-26d5e9a46606 None admin] PATH_INFO: `/v3/groups` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:29.611034 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c043ed-529e-4c96-990d-26d5e9a46606 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:29.612527 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c043ed-529e-4c96-990d-26d5e9a46606 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:29.630098 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 160/328] 10.222.0.16 () {60 vars in 1134 bytes} [Mon Sep 5 12:05:29 2022] POST /identity/v3/groups => generated 226 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:05:29.656489 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33259608-3f5b-4f26-b97c-c354d23b014e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:29.657216 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33259608-3f5b-4f26-b97c-c354d23b014e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:29.657502 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33259608-3f5b-4f26-b97c-c354d23b014e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:29.657773 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33259608-3f5b-4f26-b97c-c354d23b014e None admin] PATH_INFO: `/v3/users/37371f05b69844ecbee81b54b62a5a39` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:29.659022 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33259608-3f5b-4f26-b97c-c354d23b014e None admin] RBAC: Authorizing `identity:get_user(user_id=37371f05b69844ecbee81b54b62a5a39)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:29.660558 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33259608-3f5b-4f26-b97c-c354d23b014e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:29.662501 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 169/329] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:05:29 2022] GET /identity/v3/users/37371f05b69844ecbee81b54b62a5a39 => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:29.668200 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c043ed-529e-4c96-990d-26d5e9a46606 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:29.685510 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:29.714852 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33259608-3f5b-4f26-b97c-c354d23b014e None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:29.786106 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c043ed-529e-4c96-990d-26d5e9a46606 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:29.827234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107d283-ab76-4a90-9b25-01620a6a7e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:29.827933 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33259608-3f5b-4f26-b97c-c354d23b014e None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:29.828365 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107d283-ab76-4a90-9b25-01620a6a7e20 None admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:29.828365 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107d283-ab76-4a90-9b25-01620a6a7e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:29.828509 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107d283-ab76-4a90-9b25-01620a6a7e20 None admin] PATH_INFO: `/v3/system/users/37371f05b69844ecbee81b54b62a5a39/roles/290929f17e764f019e358e7751365fd5` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:29.830058 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107d283-ab76-4a90-9b25-01620a6a7e20 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=37371f05b69844ecbee81b54b62a5a39, role_id=290929f17e764f019e358e7751365fd5)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:29.831436 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107d283-ab76-4a90-9b25-01620a6a7e20 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:29.840568 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 161/330] 10.222.0.16 () {58 vars in 1334 bytes} [Mon Sep 5 12:05:29 2022] PUT /identity/v3/system/users/37371f05b69844ecbee81b54b62a5a39/roles/290929f17e764f019e358e7751365fd5 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:05:29.867068 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0882f-a548-4105-8d25-4adc91e8288b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:29.867785 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0882f-a548-4105-8d25-4adc91e8288b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:29.868064 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0882f-a548-4105-8d25-4adc91e8288b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:29.868337 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0882f-a548-4105-8d25-4adc91e8288b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:29.882480 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d0882f-a548-4105-8d25-4adc91e8288b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:29.887995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-04d0882f-a548-4105-8d25-4adc91e8288b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:29.887995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:29.887995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:29.887995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:29.887995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:29.887995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:29.887995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:29.887995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:29.887995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:29.887995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:29.887995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:29.887995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:29.887995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:29.887995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:29.887995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:29.887995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:29.889276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:29.889276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:29.889276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:29.889276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:29.889276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:29.889276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:29.889276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:29.889276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:29.889276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:29.889276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:29.889276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:29.889276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:29.889276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:29.889276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:29.889276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:29.889276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:29.890326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:29.890326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:29.890326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:29.890326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:29.890326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:29.890326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:29.890326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:29.890326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:29.890326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:29.890326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:29.890326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:29.890326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:29.890326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 170/331] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:05:29.896335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107d283-ab76-4a90-9b25-01620a6a7e20 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:29.991286 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:30.019158 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107d283-ab76-4a90-9b25-01620a6a7e20 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:30.058273 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347272b-16e9-4d76-bcb3-681e0f19f403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:30.058904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347272b-16e9-4d76-bcb3-681e0f19f403 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:30.059180 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347272b-16e9-4d76-bcb3-681e0f19f403 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:30.059452 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347272b-16e9-4d76-bcb3-681e0f19f403 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:30.060227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f347272b-16e9-4d76-bcb3-681e0f19f403 None admin] RBAC: Adding query filter params (name=member) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:30.060530 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f347272b-16e9-4d76-bcb3-681e0f19f403 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:30.061971 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f347272b-16e9-4d76-bcb3-681e0f19f403 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:30.071556 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 162/332] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Sep 5 12:05:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:30.077224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0882f-a548-4105-8d25-4adc91e8288b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:30.100356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:30.194138 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0882f-a548-4105-8d25-4adc91e8288b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:30.233306 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f811595-7d75-4b1f-bd9a-63712e013b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:30.233961 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f811595-7d75-4b1f-bd9a-63712e013b0c None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:30.234241 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f811595-7d75-4b1f-bd9a-63712e013b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:30.234514 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f811595-7d75-4b1f-bd9a-63712e013b0c None admin] PATH_INFO: `/v3/users/2d49d4470bbd4b5ebda6887318f8ee1c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:30.235721 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f811595-7d75-4b1f-bd9a-63712e013b0c None admin] RBAC: Authorizing `identity:get_user(user_id=2d49d4470bbd4b5ebda6887318f8ee1c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:30.237174 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f811595-7d75-4b1f-bd9a-63712e013b0c None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:30.239078 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 171/333] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:05:30 2022] GET /identity/v3/users/2d49d4470bbd4b5ebda6887318f8ee1c => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:30.244881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347272b-16e9-4d76-bcb3-681e0f19f403 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:30.386460 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347272b-16e9-4d76-bcb3-681e0f19f403 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:30.433189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee17a7f-b7b2-4a7c-895c-b08baa231f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:30.433807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee17a7f-b7b2-4a7c-895c-b08baa231f0b None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:30.434084 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee17a7f-b7b2-4a7c-895c-b08baa231f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:30.434355 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee17a7f-b7b2-4a7c-895c-b08baa231f0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:30.435208 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee17a7f-b7b2-4a7c-895c-b08baa231f0b None admin] RBAC: Adding query filter params (role.id=ede2986592354e9888dde3123b2d8c82, user.id=2d49d4470bbd4b5ebda6887318f8ee1c, scope.system=all) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:30.436520 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee17a7f-b7b2-4a7c-895c-b08baa231f0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:30.438043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee17a7f-b7b2-4a7c-895c-b08baa231f0b None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:30.443978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:30.453324 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 163/334] 10.222.0.16 () {56 vars in 1308 bytes} [Mon Sep 5 12:05:30 2022] GET /identity/v3/role_assignments?user.id=2d49d4470bbd4b5ebda6887318f8ee1c&scope.system=all&role.id=ede2986592354e9888dde3123b2d8c82 => generated 510 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:31.897100 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48613e06-88cf-4afb-ac3c-8cfcc4de58ab None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:31.898100 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48613e06-88cf-4afb-ac3c-8cfcc4de58ab None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:31.898601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48613e06-88cf-4afb-ac3c-8cfcc4de58ab None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:31.900454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 172/335] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:31.907737 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8eee5-ddbb-4326-9f92-c832024ab0ae None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:31.908659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8eee5-ddbb-4326-9f92-c832024ab0ae None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:31.909140 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8eee5-ddbb-4326-9f92-c832024ab0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:31.934580 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-76b8eee5-ddbb-4326-9f92-c832024ab0ae None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:31.936757 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-76b8eee5-ddbb-4326-9f92-c832024ab0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:32.019992 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 164/336] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:05:32.131190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0943c-2569-481e-a1b0-069bad8b76c7 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:32.131850 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0943c-2569-481e-a1b0-069bad8b76c7 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:32.132330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0943c-2569-481e-a1b0-069bad8b76c7 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:32.134191 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 173/337] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:32.139372 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d702e-4150-45ac-88bf-68e04beee654 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:32.139941 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d702e-4150-45ac-88bf-68e04beee654 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:32.140425 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d702e-4150-45ac-88bf-68e04beee654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:32.165196 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-1d1d702e-4150-45ac-88bf-68e04beee654 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:32.166716 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-1d1d702e-4150-45ac-88bf-68e04beee654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:32.248533 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 165/338] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:32.456954 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0943c-2569-481e-a1b0-069bad8b76c7 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:32.560063 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:32.578550 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0943c-2569-481e-a1b0-069bad8b76c7 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:32.621009 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7e5a6-f24b-484f-acb0-0f887e40aa3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:32.621647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7e5a6-f24b-484f-acb0-0f887e40aa3f None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:32.621929 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7e5a6-f24b-484f-acb0-0f887e40aa3f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:32.622202 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7e5a6-f24b-484f-acb0-0f887e40aa3f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:32.638479 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a7e5a6-f24b-484f-acb0-0f887e40aa3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:32.644038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-c3a7e5a6-f24b-484f-acb0-0f887e40aa3f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:32.644038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:32.644038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:32.644038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:32.644038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:32.644038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:32.644038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:32.644038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:32.644038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:32.644038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:32.644038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:32.644038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:32.644038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:32.644038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:32.644038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:32.644038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:32.644997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:32.644997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:32.644997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:32.644997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:32.644997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:32.644997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:32.644997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:32.644997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:32.644997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:32.644997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:32.644997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:32.644997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:32.644997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:32.644997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:32.644997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:32.644997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:32.645801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:32.645801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:32.645801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:32.645801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:32.645801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:32.645801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:32.645801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:32.645801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:32.645801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:32.645801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:32.645801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:32.645801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:32.645801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 174/339] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:32.653110 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d702e-4150-45ac-88bf-68e04beee654 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:32.714244 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7e5a6-f24b-484f-acb0-0f887e40aa3f None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:32.737079 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:32.796023 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d702e-4150-45ac-88bf-68e04beee654 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:32.841673 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ac190-d56e-4716-8f63-ac25c6beb311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:32.843026 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ac190-d56e-4716-8f63-ac25c6beb311 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:32.843549 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ac190-d56e-4716-8f63-ac25c6beb311 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:32.844290 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ac190-d56e-4716-8f63-ac25c6beb311 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:32.845170 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ac190-d56e-4716-8f63-ac25c6beb311 None admin] RBAC: Adding query filter params (name=member) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:32.845723 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ac190-d56e-4716-8f63-ac25c6beb311 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:32.847305 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ac190-d56e-4716-8f63-ac25c6beb311 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:32.859180 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 166/340] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Sep 5 12:05:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:32.865490 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832697c-c9f8-4bd2-8121-9c70dbb0bb33 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:32.866503 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832697c-c9f8-4bd2-8121-9c70dbb0bb33 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:32.867144 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832697c-c9f8-4bd2-8121-9c70dbb0bb33 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:32.868940 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 167/341] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:32 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:32.870606 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832697c-c9f8-4bd2-8121-9c70dbb0bb33 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:32.911956 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7e5a6-f24b-484f-acb0-0f887e40aa3f None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:32.981732 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7861b-8984-4e4e-b3f5-5e5078606491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:32.982292 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7861b-8984-4e4e-b3f5-5e5078606491 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:32.982589 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7861b-8984-4e4e-b3f5-5e5078606491 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:32.982863 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7861b-8984-4e4e-b3f5-5e5078606491 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:33.008817 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a7861b-8984-4e4e-b3f5-5e5078606491 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:33.013590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-f1a7861b-8984-4e4e-b3f5-5e5078606491 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:05:33.013590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:33.013590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:33.013590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:33.013590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:33.013590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:33.013590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:33.013590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:33.013590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:33.013590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:33.013590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:33.013590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:33.013590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:33.013590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:33.013590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:33.013590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:33.015051 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:33.015051 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:33.015051 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:33.015051 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:33.015051 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:33.015051 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:33.015051 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:33.015051 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:33.015051 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:33.015051 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:33.015051 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:33.015051 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:33.015051 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:33.015051 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:33.015051 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:33.015051 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:33.015892 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:33.015892 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:33.015892 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:33.015892 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:33.015892 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:33.015892 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:33.015892 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:33.015892 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:33.015892 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:33.015892 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:33.015892 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:05:33.015892 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:33.015892 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 175/342] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:05:33.018865 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75e5cb-e2fc-4bff-adb3-26f875fab32a None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:33.020416 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75e5cb-e2fc-4bff-adb3-26f875fab32a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:33.021364 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75e5cb-e2fc-4bff-adb3-26f875fab32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:33.038712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:33.056674 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-ad75e5cb-e2fc-4bff-adb3-26f875fab32a None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:33.057853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ad75e5cb-e2fc-4bff-adb3-26f875fab32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:33.064926 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832697c-c9f8-4bd2-8121-9c70dbb0bb33 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:33.092880 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:33.124804 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c1bbc-7d75-4f5b-b5ee-e13b1e751500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:33.125418 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c1bbc-7d75-4f5b-b5ee-e13b1e751500 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:33.125690 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c1bbc-7d75-4f5b-b5ee-e13b1e751500 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:33.125959 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c1bbc-7d75-4f5b-b5ee-e13b1e751500 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:33.186566 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 176/343] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:33.191911 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad75e5cb-e2fc-4bff-adb3-26f875fab32a None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:33.239506 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c1bbc-7d75-4f5b-b5ee-e13b1e751500 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:33.240970 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c1bbc-7d75-4f5b-b5ee-e13b1e751500 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:33.316184 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad75e5cb-e2fc-4bff-adb3-26f875fab32a None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:33.360036 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 168/344] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:05:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 05 12:05:33.361390 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5b7e0-7a53-42e0-914d-58ea4b1c14e3 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:33.361675 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5b7e0-7a53-42e0-914d-58ea4b1c14e3 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:33.361945 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5b7e0-7a53-42e0-914d-58ea4b1c14e3 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:33.362232 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeada084-8379-4721-8474-f34cca1b4cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:33.363057 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeada084-8379-4721-8474-f34cca1b4cfb None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:33.363449 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 169/345] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:33.363967 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeada084-8379-4721-8474-f34cca1b4cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:33.364428 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeada084-8379-4721-8474-f34cca1b4cfb None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:33.365349 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeada084-8379-4721-8474-f34cca1b4cfb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:33.365866 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeada084-8379-4721-8474-f34cca1b4cfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:33.367996 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeada084-8379-4721-8474-f34cca1b4cfb None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:33.368721 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04147bcc-edba-457e-a433-03bb3d698e8f None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:33.369003 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04147bcc-edba-457e-a433-03bb3d698e8f None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:33.369272 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04147bcc-edba-457e-a433-03bb3d698e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:33.370501 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:33.390025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 177/346] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Sep 5 12:05:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:33.390237 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-04147bcc-edba-457e-a433-03bb3d698e8f None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:33.390886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeada084-8379-4721-8474-f34cca1b4cfb None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:33.391478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-04147bcc-edba-457e-a433-03bb3d698e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:33.407913 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:33.471315 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 170/347] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:05:33.472296 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04147bcc-edba-457e-a433-03bb3d698e8f None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:33.548808 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeada084-8379-4721-8474-f34cca1b4cfb None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:33.598256 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52665d-a8ed-4cce-8894-50bb940af608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:33.598897 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52665d-a8ed-4cce-8894-50bb940af608 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:33.599262 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52665d-a8ed-4cce-8894-50bb940af608 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:33.599560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52665d-a8ed-4cce-8894-50bb940af608 None admin] PATH_INFO: `/v3/projects/405d2cd2fe68456ca034a57f24f6b015` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:33.603127 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52665d-a8ed-4cce-8894-50bb940af608 None admin] RBAC: Authorizing `identity:get_project(project_id=405d2cd2fe68456ca034a57f24f6b015)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:33.604623 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52665d-a8ed-4cce-8894-50bb940af608 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:33.606624 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 178/348] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:33 2022] GET /identity/v3/projects/405d2cd2fe68456ca034a57f24f6b015 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:33.612823 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52665d-a8ed-4cce-8894-50bb940af608 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:33.688923 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:33.707816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04147bcc-edba-457e-a433-03bb3d698e8f None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:33.727513 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52665d-a8ed-4cce-8894-50bb940af608 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:33.752809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375171d-dffc-47cc-bd50-98a50133b39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:33.753442 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375171d-dffc-47cc-bd50-98a50133b39f None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:33.753716 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375171d-dffc-47cc-bd50-98a50133b39f None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:33.753982 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375171d-dffc-47cc-bd50-98a50133b39f None admin] PATH_INFO: `/v3/users/37371f05b69844ecbee81b54b62a5a39` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:33.755114 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375171d-dffc-47cc-bd50-98a50133b39f None admin] RBAC: Authorizing `identity:get_user(user_id=37371f05b69844ecbee81b54b62a5a39)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:33.756605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375171d-dffc-47cc-bd50-98a50133b39f None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:33.758197 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 171/349] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:05:33 2022] GET /identity/v3/users/37371f05b69844ecbee81b54b62a5a39 => generated 293 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:33.764355 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375171d-dffc-47cc-bd50-98a50133b39f None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:33.783217 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b13311-7d0c-4bde-a846-d4d18ae85dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:33.784019 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b13311-7d0c-4bde-a846-d4d18ae85dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:33.784303 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b13311-7d0c-4bde-a846-d4d18ae85dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:33.784583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b13311-7d0c-4bde-a846-d4d18ae85dc7 None admin] PATH_INFO: `/v3/groups/9bba5ca03ec949a287b22ba2f431d6cc` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:33.786269 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:33.796853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b13311-7d0c-4bde-a846-d4d18ae85dc7 None admin] RBAC: Authorizing `identity:get_group(group_id=9bba5ca03ec949a287b22ba2f431d6cc)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:33.798318 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b13311-7d0c-4bde-a846-d4d18ae85dc7 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:33.800405 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 179/350] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Sep 5 12:05:33 2022] GET /identity/v3/groups/9bba5ca03ec949a287b22ba2f431d6cc => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:33.806555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b13311-7d0c-4bde-a846-d4d18ae85dc7 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:33.878938 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375171d-dffc-47cc-bd50-98a50133b39f None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:33.917375 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7e285-9400-452f-80b2-b08580c8124d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:33.917980 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7e285-9400-452f-80b2-b08580c8124d None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:33.918254 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7e285-9400-452f-80b2-b08580c8124d None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:33.918521 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7e285-9400-452f-80b2-b08580c8124d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:33.919251 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f7e285-9400-452f-80b2-b08580c8124d None admin] RBAC: Adding query filter params (role.id=290929f17e764f019e358e7751365fd5, user.id=37371f05b69844ecbee81b54b62a5a39, scope.system=all) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:33.919600 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f7e285-9400-452f-80b2-b08580c8124d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:33.921080 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f7e285-9400-452f-80b2-b08580c8124d None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:33.921298 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b13311-7d0c-4bde-a846-d4d18ae85dc7 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:33.936168 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 172/351] 10.222.0.16 () {56 vars in 1308 bytes} [Mon Sep 5 12:05:33 2022] GET /identity/v3/role_assignments?user.id=37371f05b69844ecbee81b54b62a5a39&scope.system=all&role.id=290929f17e764f019e358e7751365fd5 => generated 510 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:33.937412 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799321a-053b-4314-bf41-aec7145a3a63 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:33.937691 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799321a-053b-4314-bf41-aec7145a3a63 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:33.937958 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799321a-053b-4314-bf41-aec7145a3a63 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:33.939154 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 173/352] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:33.944160 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1799321a-053b-4314-bf41-aec7145a3a63 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:33.952195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:33.961263 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d591a-8c4a-4b25-bfb2-5419819933be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:33.962005 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d591a-8c4a-4b25-bfb2-5419819933be None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:33.962295 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d591a-8c4a-4b25-bfb2-5419819933be None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:33.962577 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d591a-8c4a-4b25-bfb2-5419819933be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:33.963331 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d591a-8c4a-4b25-bfb2-5419819933be None admin] RBAC: Adding query filter params (group.id=9bba5ca03ec949a287b22ba2f431d6cc, role.id=ede2986592354e9888dde3123b2d8c82, scope.project.id=405d2cd2fe68456ca034a57f24f6b015) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:33.963689 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d591a-8c4a-4b25-bfb2-5419819933be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:33.965221 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d591a-8c4a-4b25-bfb2-5419819933be None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:33.973465 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 180/353] 10.222.0.16 () {56 vars in 1376 bytes} [Mon Sep 5 12:05:33 2022] GET /identity/v3/role_assignments?group.id=9bba5ca03ec949a287b22ba2f431d6cc&scope.project.id=405d2cd2fe68456ca034a57f24f6b015&role.id=ede2986592354e9888dde3123b2d8c82 => generated 260 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:33.974810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f479a2-be38-4c5a-a65e-a19a0cda6261 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:33.975100 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f479a2-be38-4c5a-a65e-a19a0cda6261 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:33.975379 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f479a2-be38-4c5a-a65e-a19a0cda6261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:34.014995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-44f479a2-be38-4c5a-a65e-a19a0cda6261 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:34.016312 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-44f479a2-be38-4c5a-a65e-a19a0cda6261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:34.088439 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:34.152297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1799321a-053b-4314-bf41-aec7145a3a63 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:34.162791 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 181/354] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:34.164108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f479a2-be38-4c5a-a65e-a19a0cda6261 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:34.226635 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7928c520-cc43-4474-b427-3fd54d5d0f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:34.227286 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928c520-cc43-4474-b427-3fd54d5d0f51 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:34.227634 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928c520-cc43-4474-b427-3fd54d5d0f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:34.227913 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928c520-cc43-4474-b427-3fd54d5d0f51 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:34.246758 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7928c520-cc43-4474-b427-3fd54d5d0f51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:34.253064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-7928c520-cc43-4474-b427-3fd54d5d0f51 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:05:34.253064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:34.253064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:34.253064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:34.253064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:34.253064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:34.253064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:34.253064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:34.253064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:34.253064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:34.253064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:34.253064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:34.253064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:34.253064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:34.253064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:34.253064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:34.254051 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:34.254051 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:34.254051 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:34.254051 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:34.254051 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:34.254051 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:34.254051 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:34.254051 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:34.254051 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:34.254051 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:34.254051 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:34.254051 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:34.254051 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:34.254051 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:34.254051 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:34.254051 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:34.254876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:34.254876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:34.254876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:34.254876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:34.254876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:34.254876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:34.254876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:34.254876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:34.254876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:34.254876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:34.254876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:05:34.254876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:34.254876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 174/355] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Sep 5 12:05:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:34.260593 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7928c520-cc43-4474-b427-3fd54d5d0f51 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:34.309029 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f479a2-be38-4c5a-a65e-a19a0cda6261 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:34.334792 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:34.348294 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadb540c-5290-4abf-9ac5-9cbd6e26f411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:34.348844 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb540c-5290-4abf-9ac5-9cbd6e26f411 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:34.349122 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb540c-5290-4abf-9ac5-9cbd6e26f411 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:34.349391 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb540c-5290-4abf-9ac5-9cbd6e26f411 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:34.362417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadb540c-5290-4abf-9ac5-9cbd6e26f411 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:34.367230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-eadb540c-5290-4abf-9ac5-9cbd6e26f411 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:34.367230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:34.367230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:34.367230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:34.367230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:34.367230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:34.367230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:34.367230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:34.367230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:34.367230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:34.367230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:34.367230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:34.367230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:34.367230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:34.367230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:34.367230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:34.368182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:34.368182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:34.368182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:34.368182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:34.368182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:34.368182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:34.368182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:34.368182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:34.368182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:34.368182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:34.368182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:34.368182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:34.368182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:34.368182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:34.368182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:34.368182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:34.369003 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:34.369003 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:34.369003 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:34.369003 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:34.369003 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:34.369003 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:34.369003 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:34.369003 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:34.369003 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:34.369003 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:34.369003 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:34.369003 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:34.369003 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 182/356] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:34.378006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadb540c-5290-4abf-9ac5-9cbd6e26f411 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:34.439044 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7928c520-cc43-4474-b427-3fd54d5d0f51 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:34.481718 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8bdceb-597f-4331-8b00-c9b4cc953d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:34.482592 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8bdceb-597f-4331-8b00-c9b4cc953d70 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:34.482871 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8bdceb-597f-4331-8b00-c9b4cc953d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:34.483144 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8bdceb-597f-4331-8b00-c9b4cc953d70 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:34.483852 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8bdceb-597f-4331-8b00-c9b4cc953d70 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:34.484142 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8bdceb-597f-4331-8b00-c9b4cc953d70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:34.486061 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8bdceb-597f-4331-8b00-c9b4cc953d70 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:34.488752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:34.499634 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 175/357] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:05:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:34.505899 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8bdceb-597f-4331-8b00-c9b4cc953d70 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:34.556267 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadb540c-5290-4abf-9ac5-9cbd6e26f411 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:34.601447 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8016cd-b128-4312-957d-73c4edeac179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:34.601972 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8016cd-b128-4312-957d-73c4edeac179 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:34.602264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8016cd-b128-4312-957d-73c4edeac179 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:34.602525 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8016cd-b128-4312-957d-73c4edeac179 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:34.603162 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8016cd-b128-4312-957d-73c4edeac179 None admin] RBAC: Adding query filter params (name=member) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:34.603506 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8016cd-b128-4312-957d-73c4edeac179 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:34.604945 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8016cd-b128-4312-957d-73c4edeac179 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:34.617575 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 183/358] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Sep 5 12:05:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:34.618853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2556f12-206d-4052-91f4-c5c02b15ae5f None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:34.619120 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2556f12-206d-4052-91f4-c5c02b15ae5f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:34.619394 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2556f12-206d-4052-91f4-c5c02b15ae5f None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:34.620783 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 184/359] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:34.621337 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2556f12-206d-4052-91f4-c5c02b15ae5f None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:34.650592 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8bdceb-597f-4331-8b00-c9b4cc953d70 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:34.694924 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:34.700413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81697e-6d6c-40a2-a2e7-f3083a022df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:34.701482 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81697e-6d6c-40a2-a2e7-f3083a022df1 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:34.701762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81697e-6d6c-40a2-a2e7-f3083a022df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:34.702034 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81697e-6d6c-40a2-a2e7-f3083a022df1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:34.786202 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:34.789566 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2556f12-206d-4052-91f4-c5c02b15ae5f None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:34.796541 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81697e-6d6c-40a2-a2e7-f3083a022df1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:34.798019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81697e-6d6c-40a2-a2e7-f3083a022df1 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:34.853554 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046b56f-904e-481e-af5b-55c49c6725a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:34.854728 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046b56f-904e-481e-af5b-55c49c6725a5 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:34.855539 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046b56f-904e-481e-af5b-55c49c6725a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:34.855991 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046b56f-904e-481e-af5b-55c49c6725a5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:34.870992 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046b56f-904e-481e-af5b-55c49c6725a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:34.876777 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-9046b56f-904e-481e-af5b-55c49c6725a5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:05:34.876777 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:34.876777 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:34.876777 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:34.876777 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:34.876777 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:34.876777 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:34.876777 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:34.876777 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:34.876777 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:34.876777 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:34.876777 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:34.876777 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:34.876777 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:34.876777 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:34.876777 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:34.877710 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:34.877710 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:34.877710 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:34.877710 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:34.877710 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:34.877710 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:34.877710 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:34.877710 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:34.877710 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:34.877710 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:34.877710 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:34.877710 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:34.877710 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:34.877710 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:34.877710 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:34.877710 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:34.878510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:34.878510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:34.878510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:34.878510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:34.878510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:34.878510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:34.878510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:34.878510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:34.878510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:34.878510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:34.878510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:05:34.878510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:34.880806 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 185/360] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Sep 5 12:05:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:34.881823 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046b56f-904e-481e-af5b-55c49c6725a5 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:34.939066 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 176/361] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:05:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 05 12:05:34.939066 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b94bd-ee34-4a51-80f0-2bbbb1d50a12 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:34.939066 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b94bd-ee34-4a51-80f0-2bbbb1d50a12 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:34.939066 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b94bd-ee34-4a51-80f0-2bbbb1d50a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:34.963117 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-3c0b94bd-ee34-4a51-80f0-2bbbb1d50a12 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:34.964357 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-3c0b94bd-ee34-4a51-80f0-2bbbb1d50a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:35.030180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046b56f-904e-481e-af5b-55c49c6725a5 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:35.034651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:35.063624 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 177/362] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:35.066077 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b94bd-ee34-4a51-80f0-2bbbb1d50a12 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:35.084016 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07a543-bedd-49ae-b820-9edb641e54c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:35.084936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07a543-bedd-49ae-b820-9edb641e54c6 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:35.085219 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07a543-bedd-49ae-b820-9edb641e54c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:35.085524 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07a543-bedd-49ae-b820-9edb641e54c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:35.101127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:35.181429 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07a543-bedd-49ae-b820-9edb641e54c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:35.183120 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07a543-bedd-49ae-b820-9edb641e54c6 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:35.201556 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b94bd-ee34-4a51-80f0-2bbbb1d50a12 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:35.256980 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45c223-ce23-44e9-9624-73a03389e36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:35.257618 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45c223-ce23-44e9-9624-73a03389e36a None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:35.258158 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45c223-ce23-44e9-9624-73a03389e36a None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:35.258468 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45c223-ce23-44e9-9624-73a03389e36a None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:35.259302 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45c223-ce23-44e9-9624-73a03389e36a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:35.263083 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45c223-ce23-44e9-9624-73a03389e36a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:35.263083 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45c223-ce23-44e9-9624-73a03389e36a None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:35.278503 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 178/363] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:05:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:35.283181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45c223-ce23-44e9-9624-73a03389e36a None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:35.319610 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 186/364] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:05:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:05:35.320757 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07a543-bedd-49ae-b820-9edb641e54c6 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:35.325048 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:35.417085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:35.433755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45c223-ce23-44e9-9624-73a03389e36a None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:35.455932 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07a543-bedd-49ae-b820-9edb641e54c6 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:35.479586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b8359-962f-4e8f-9f64-abf049b315af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:35.480548 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b8359-962f-4e8f-9f64-abf049b315af None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:35.481005 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b8359-962f-4e8f-9f64-abf049b315af None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:35.481431 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b8359-962f-4e8f-9f64-abf049b315af None admin] PATH_INFO: `/v3/projects/405d2cd2fe68456ca034a57f24f6b015` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:35.483169 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b8359-962f-4e8f-9f64-abf049b315af None admin] RBAC: Authorizing `identity:get_project(project_id=405d2cd2fe68456ca034a57f24f6b015)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:35.485692 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b8359-962f-4e8f-9f64-abf049b315af None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:35.486519 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 179/365] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:35 2022] GET /identity/v3/projects/405d2cd2fe68456ca034a57f24f6b015 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:35.487129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b8359-962f-4e8f-9f64-abf049b315af None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:35.512045 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa7737c-82d0-40a1-bc64-40b215eab016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:35.513385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7737c-82d0-40a1-bc64-40b215eab016 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:35.513915 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7737c-82d0-40a1-bc64-40b215eab016 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:35.514422 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7737c-82d0-40a1-bc64-40b215eab016 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:35.618873 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa7737c-82d0-40a1-bc64-40b215eab016 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:35.620560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa7737c-82d0-40a1-bc64-40b215eab016 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:35.624431 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:35.633010 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b8359-962f-4e8f-9f64-abf049b315af None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:35.681455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a688122-a022-40b1-8dfe-255f3c287391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:35.682372 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a688122-a022-40b1-8dfe-255f3c287391 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:35.682656 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a688122-a022-40b1-8dfe-255f3c287391 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:35.682926 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a688122-a022-40b1-8dfe-255f3c287391 None admin] PATH_INFO: `/v3/projects/75434cc693884f639e8b84015d47cc58` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:35.684346 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a688122-a022-40b1-8dfe-255f3c287391 None admin] RBAC: Authorizing `identity:get_project(project_id=75434cc693884f639e8b84015d47cc58)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:35.687750 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a688122-a022-40b1-8dfe-255f3c287391 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:35.690519 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 180/366] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:35 2022] GET /identity/v3/projects/75434cc693884f639e8b84015d47cc58 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:35.691088 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a688122-a022-40b1-8dfe-255f3c287391 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:35.725343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:35.764417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 187/367] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:05:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:05:35.764417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa7737c-82d0-40a1-bc64-40b215eab016 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:35.846355 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a688122-a022-40b1-8dfe-255f3c287391 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:35.894067 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03e05d-3171-4dac-b8d6-6f3fe8c58d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:35.895022 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03e05d-3171-4dac-b8d6-6f3fe8c58d77 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:35.895402 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03e05d-3171-4dac-b8d6-6f3fe8c58d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:35.895713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03e05d-3171-4dac-b8d6-6f3fe8c58d77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:35.914579 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03e05d-3171-4dac-b8d6-6f3fe8c58d77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:35.919464 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa7737c-82d0-40a1-bc64-40b215eab016 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:35.920833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-aa03e05d-3171-4dac-b8d6-6f3fe8c58d77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:05:35.920833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:35.920833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:35.920833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:35.920833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:35.920833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:35.920833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:35.920833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:35.920833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:35.920833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:35.920833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:35.920833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:35.920833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:35.920833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:35.920833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:35.920833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:35.921738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:35.921738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:35.921738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:35.921738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:35.921738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:35.921738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:35.921738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:35.921738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:35.921738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:35.921738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:35.921738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:35.921738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:35.921738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:35.921738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:35.921738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:35.921738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:35.922574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:35.922574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:35.922574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:35.922574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:35.922574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:35.922574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:35.922574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:35.922574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:35.922574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:35.922574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:35.922574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:05:35.922574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:35.922574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 181/368] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:05:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:35.923259 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03e05d-3171-4dac-b8d6-6f3fe8c58d77 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:35.928564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:35.976216 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cef1e-cdf4-4910-a8db-f944ff087c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:35.976537 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cef1e-cdf4-4910-a8db-f944ff087c08 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:35.976780 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cef1e-cdf4-4910-a8db-f944ff087c08 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:35.977069 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cef1e-cdf4-4910-a8db-f944ff087c08 None admin] PATH_INFO: `/v3/groups/9bba5ca03ec949a287b22ba2f431d6cc` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:35.981939 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49cef1e-cdf4-4910-a8db-f944ff087c08 None admin] RBAC: Authorizing `identity:get_group(group_id=9bba5ca03ec949a287b22ba2f431d6cc)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:35.981939 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49cef1e-cdf4-4910-a8db-f944ff087c08 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:35.983112 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 188/369] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Sep 5 12:05:35 2022] GET /identity/v3/groups/9bba5ca03ec949a287b22ba2f431d6cc => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:35.985722 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cef1e-cdf4-4910-a8db-f944ff087c08 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:36.043579 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:36.063736 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03e05d-3171-4dac-b8d6-6f3fe8c58d77 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:36.107154 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ecc61-66fd-44ca-a1fe-ef04dd9587da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:36.108537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ecc61-66fd-44ca-a1fe-ef04dd9587da None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:36.108915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ecc61-66fd-44ca-a1fe-ef04dd9587da None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:36.110269 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ecc61-66fd-44ca-a1fe-ef04dd9587da None admin] PATH_INFO: `/v3/groups/9bba5ca03ec949a287b22ba2f431d6cc` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:36.112169 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ecc61-66fd-44ca-a1fe-ef04dd9587da None admin] RBAC: Authorizing `identity:get_group(group_id=9bba5ca03ec949a287b22ba2f431d6cc)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:36.113294 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ecc61-66fd-44ca-a1fe-ef04dd9587da None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:36.119570 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 182/370] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Sep 5 12:05:35 2022] GET /identity/v3/groups/9bba5ca03ec949a287b22ba2f431d6cc => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:36.119570 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ecc61-66fd-44ca-a1fe-ef04dd9587da None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:36.131119 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cef1e-cdf4-4910-a8db-f944ff087c08 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:36.192907 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3656b4-50e8-41ed-9043-917d15f4c902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:36.193545 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3656b4-50e8-41ed-9043-917d15f4c902 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:36.193823 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3656b4-50e8-41ed-9043-917d15f4c902 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:36.194092 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3656b4-50e8-41ed-9043-917d15f4c902 None admin] PATH_INFO: `/v3/projects/75434cc693884f639e8b84015d47cc58` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:36.195238 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3656b4-50e8-41ed-9043-917d15f4c902 None admin] RBAC: Authorizing `identity:get_project(project_id=75434cc693884f639e8b84015d47cc58)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:36.196712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3656b4-50e8-41ed-9043-917d15f4c902 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:36.198589 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 189/371] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:35 2022] GET /identity/v3/projects/75434cc693884f639e8b84015d47cc58 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:36.200408 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3656b4-50e8-41ed-9043-917d15f4c902 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:36.256066 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ecc61-66fd-44ca-a1fe-ef04dd9587da None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:36.260443 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:36.305466 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59a120-4d8f-41cb-ac64-84c85cfed345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:36.306590 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59a120-4d8f-41cb-ac64-84c85cfed345 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:36.306590 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59a120-4d8f-41cb-ac64-84c85cfed345 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:36.306590 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59a120-4d8f-41cb-ac64-84c85cfed345 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:36.306989 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59a120-4d8f-41cb-ac64-84c85cfed345 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:36.307289 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59a120-4d8f-41cb-ac64-84c85cfed345 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:36.308993 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59a120-4d8f-41cb-ac64-84c85cfed345 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:36.319909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 183/372] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:36.321230 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59a120-4d8f-41cb-ac64-84c85cfed345 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:36.338932 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3656b4-50e8-41ed-9043-917d15f4c902 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:36.340077 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:36.382701 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02906e1-6c40-407e-b821-c0e60a3dc6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:36.383391 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02906e1-6c40-407e-b821-c0e60a3dc6cd None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:36.383782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02906e1-6c40-407e-b821-c0e60a3dc6cd None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:36.384051 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02906e1-6c40-407e-b821-c0e60a3dc6cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:36.385134 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02906e1-6c40-407e-b821-c0e60a3dc6cd None admin] RBAC: Adding query filter params (group.id=9bba5ca03ec949a287b22ba2f431d6cc, role.id=20bf8c77837d43b590a2bd9a67d01f84, scope.project.id=405d2cd2fe68456ca034a57f24f6b015) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:36.385134 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02906e1-6c40-407e-b821-c0e60a3dc6cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:36.386502 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02906e1-6c40-407e-b821-c0e60a3dc6cd None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:36.393918 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 190/373] 10.222.0.16 () {56 vars in 1376 bytes} [Mon Sep 5 12:05:36 2022] GET /identity/v3/role_assignments?group.id=9bba5ca03ec949a287b22ba2f431d6cc&scope.project.id=405d2cd2fe68456ca034a57f24f6b015&role.id=20bf8c77837d43b590a2bd9a67d01f84 => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:36.396735 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02906e1-6c40-407e-b821-c0e60a3dc6cd None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:36.447352 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59a120-4d8f-41cb-ac64-84c85cfed345 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:36.494313 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78369ae8-f50f-4308-b6d5-86c05369433b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:36.494943 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78369ae8-f50f-4308-b6d5-86c05369433b None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:36.495222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78369ae8-f50f-4308-b6d5-86c05369433b None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:36.495498 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78369ae8-f50f-4308-b6d5-86c05369433b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:36.496315 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78369ae8-f50f-4308-b6d5-86c05369433b None admin] RBAC: Adding query filter params (group.id=9bba5ca03ec949a287b22ba2f431d6cc, role.id=ede2986592354e9888dde3123b2d8c82, scope.project.id=75434cc693884f639e8b84015d47cc58) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:36.496630 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78369ae8-f50f-4308-b6d5-86c05369433b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:36.498131 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78369ae8-f50f-4308-b6d5-86c05369433b None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:36.504417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 184/374] 10.222.0.16 () {56 vars in 1376 bytes} [Mon Sep 5 12:05:36 2022] GET /identity/v3/role_assignments?group.id=9bba5ca03ec949a287b22ba2f431d6cc&scope.project.id=75434cc693884f639e8b84015d47cc58&role.id=ede2986592354e9888dde3123b2d8c82 => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:36.506724 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec5cb2-6baa-4a8b-86e3-f9020f41e83a None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:36.506966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec5cb2-6baa-4a8b-86e3-f9020f41e83a None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:36.507221 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec5cb2-6baa-4a8b-86e3-f9020f41e83a None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:36.509024 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 185/375] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:36.509888 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec5cb2-6baa-4a8b-86e3-f9020f41e83a None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:36.547583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02906e1-6c40-407e-b821-c0e60a3dc6cd None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:36.550681 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:36.602235 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836fcffb-c008-4147-8f15-3642233fade6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:36.603286 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fcffb-c008-4147-8f15-3642233fade6 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:36.604026 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fcffb-c008-4147-8f15-3642233fade6 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:36.604499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fcffb-c008-4147-8f15-3642233fade6 None admin] PATH_INFO: `/v3/groups/9bba5ca03ec949a287b22ba2f431d6cc` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:36.606046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836fcffb-c008-4147-8f15-3642233fade6 None admin] RBAC: Authorizing `identity:get_group(group_id=9bba5ca03ec949a287b22ba2f431d6cc)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:36.608147 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836fcffb-c008-4147-8f15-3642233fade6 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:36.610417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 191/376] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Sep 5 12:05:36 2022] GET /identity/v3/groups/9bba5ca03ec949a287b22ba2f431d6cc => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:36.611957 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344f278-6230-489a-ab16-3d44aaaa8a7b None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:36.612467 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344f278-6230-489a-ab16-3d44aaaa8a7b None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:36.612930 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344f278-6230-489a-ab16-3d44aaaa8a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:36.633259 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-c344f278-6230-489a-ab16-3d44aaaa8a7b None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:36.634447 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:36.635843 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c344f278-6230-489a-ab16-3d44aaaa8a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:36.662710 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec5cb2-6baa-4a8b-86e3-f9020f41e83a None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:36.704116 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39f11c-e635-4c01-b350-0302ed235d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:36.704843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39f11c-e635-4c01-b350-0302ed235d56 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:36.705111 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39f11c-e635-4c01-b350-0302ed235d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:36.705387 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39f11c-e635-4c01-b350-0302ed235d56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:36.718626 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 192/377] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:36.719827 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344f278-6230-489a-ab16-3d44aaaa8a7b None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:36.850074 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39f11c-e635-4c01-b350-0302ed235d56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:36.851584 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39f11c-e635-4c01-b350-0302ed235d56 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:36.950509 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344f278-6230-489a-ab16-3d44aaaa8a7b None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:36.973049 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:36.991900 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:37.005030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef47bc5-8ae6-40e9-8e59-856ab9126ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:37.005812 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef47bc5-8ae6-40e9-8e59-856ab9126ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:37.006113 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef47bc5-8ae6-40e9-8e59-856ab9126ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:37.006374 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef47bc5-8ae6-40e9-8e59-856ab9126ec6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:37.007116 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef47bc5-8ae6-40e9-8e59-856ab9126ec6 None admin] RBAC: Adding query filter params (group.id=9bba5ca03ec949a287b22ba2f431d6cc, role.id=20bf8c77837d43b590a2bd9a67d01f84, scope.project.id=75434cc693884f639e8b84015d47cc58) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:37.007429 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef47bc5-8ae6-40e9-8e59-856ab9126ec6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:37.008910 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef47bc5-8ae6-40e9-8e59-856ab9126ec6 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:37.014788 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 193/378] 10.222.0.16 () {56 vars in 1376 bytes} [Mon Sep 5 12:05:36 2022] GET /identity/v3/role_assignments?group.id=9bba5ca03ec949a287b22ba2f431d6cc&scope.project.id=75434cc693884f639e8b84015d47cc58&role.id=20bf8c77837d43b590a2bd9a67d01f84 => generated 260 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:37.040792 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 186/379] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:05:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:05:37.047369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef47bc5-8ae6-40e9-8e59-856ab9126ec6 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:37.073747 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39f11c-e635-4c01-b350-0302ed235d56 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:37.184907 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39f11c-e635-4c01-b350-0302ed235d56 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:37.222809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82615ea8-ce36-4929-ae31-6f3a547976f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:37.223872 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82615ea8-ce36-4929-ae31-6f3a547976f7 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:37.224345 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82615ea8-ce36-4929-ae31-6f3a547976f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:37.224803 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82615ea8-ce36-4929-ae31-6f3a547976f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:37.237996 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82615ea8-ce36-4929-ae31-6f3a547976f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:37.244087 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-82615ea8-ce36-4929-ae31-6f3a547976f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:37.244087 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:37.244087 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:37.244087 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:37.244087 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:37.244087 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:37.244087 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:37.244087 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:37.244087 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:37.244087 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:37.244087 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:37.244087 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:37.244087 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:37.244087 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:37.244087 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:37.244087 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:37.245049 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:37.245049 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:37.245049 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:37.245049 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:37.245049 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:37.245049 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:37.245049 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:37.245049 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:37.245049 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:37.245049 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:37.245049 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:37.245049 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:37.245049 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:37.245049 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:37.245049 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:37.245049 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:37.245855 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:37.245855 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:37.245855 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:37.245855 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:37.245855 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:37.245855 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:37.245855 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:37.245855 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:37.245855 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:37.245855 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:37.245855 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:37.245855 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:37.256567 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 187/380] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:05:37.259330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef47bc5-8ae6-40e9-8e59-856ab9126ec6 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:37.264363 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82615ea8-ce36-4929-ae31-6f3a547976f7 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:37.324747 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a678285-f7c2-4807-ad61-6cb575dbf8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:37.325428 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a678285-f7c2-4807-ad61-6cb575dbf8dd None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:37.325709 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a678285-f7c2-4807-ad61-6cb575dbf8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:37.325985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a678285-f7c2-4807-ad61-6cb575dbf8dd None admin] PATH_INFO: `/v3/projects/898d4110c367417eb23c75ef5e13bda3` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:37.328220 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a678285-f7c2-4807-ad61-6cb575dbf8dd None admin] RBAC: Authorizing `identity:get_project(project_id=898d4110c367417eb23c75ef5e13bda3)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:37.329684 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a678285-f7c2-4807-ad61-6cb575dbf8dd None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:37.332273 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 194/381] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:37 2022] GET /identity/v3/projects/898d4110c367417eb23c75ef5e13bda3 => generated 344 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:37.339212 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:37.339472 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a678285-f7c2-4807-ad61-6cb575dbf8dd None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:37.391930 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:37.470856 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a678285-f7c2-4807-ad61-6cb575dbf8dd None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:37.497516 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82615ea8-ce36-4929-ae31-6f3a547976f7 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:37.514700 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2b100-fbe5-4555-aa6d-c3bd6bf006de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:37.515345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2b100-fbe5-4555-aa6d-c3bd6bf006de None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:37.515654 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2b100-fbe5-4555-aa6d-c3bd6bf006de None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:37.515931 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2b100-fbe5-4555-aa6d-c3bd6bf006de None admin] PATH_INFO: `/v3/groups/335a64d918d94be1a91101811973a892` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:37.528016 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b2b100-fbe5-4555-aa6d-c3bd6bf006de None admin] RBAC: Authorizing `identity:get_group(group_id=335a64d918d94be1a91101811973a892)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:37.531134 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b2b100-fbe5-4555-aa6d-c3bd6bf006de None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:37.532883 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 195/382] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Sep 5 12:05:37 2022] GET /identity/v3/groups/335a64d918d94be1a91101811973a892 => generated 229 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:37.545724 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2b100-fbe5-4555-aa6d-c3bd6bf006de None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:37.573982 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb114cce-63de-4f9b-a21f-5ea2c55c7219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:37.574600 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb114cce-63de-4f9b-a21f-5ea2c55c7219 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:37.574881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb114cce-63de-4f9b-a21f-5ea2c55c7219 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:37.575156 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb114cce-63de-4f9b-a21f-5ea2c55c7219 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:37.575869 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb114cce-63de-4f9b-a21f-5ea2c55c7219 None admin] RBAC: Adding query filter params (name=member) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:37.576209 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb114cce-63de-4f9b-a21f-5ea2c55c7219 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:37.577614 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb114cce-63de-4f9b-a21f-5ea2c55c7219 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:37.599493 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 188/383] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Sep 5 12:05:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:37.603091 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb114cce-63de-4f9b-a21f-5ea2c55c7219 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:37.694831 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2b100-fbe5-4555-aa6d-c3bd6bf006de None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:37.707908 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:37.771939 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7841dfeb-8050-4c58-885d-e8fa9c62daf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:37.772482 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841dfeb-8050-4c58-885d-e8fa9c62daf7 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:37.772763 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841dfeb-8050-4c58-885d-e8fa9c62daf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:37.773035 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841dfeb-8050-4c58-885d-e8fa9c62daf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:37.773775 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7841dfeb-8050-4c58-885d-e8fa9c62daf7 None admin] RBAC: Adding query filter params (group.id=335a64d918d94be1a91101811973a892, role.id=618c48706f7f445abeddf7eb2f8e927a, scope.project.id=898d4110c367417eb23c75ef5e13bda3) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:37.774081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7841dfeb-8050-4c58-885d-e8fa9c62daf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:37.775564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7841dfeb-8050-4c58-885d-e8fa9c62daf7 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:37.795238 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 196/384] 10.222.0.16 () {56 vars in 1376 bytes} [Mon Sep 5 12:05:37 2022] GET /identity/v3/role_assignments?group.id=335a64d918d94be1a91101811973a892&scope.project.id=898d4110c367417eb23c75ef5e13bda3&role.id=618c48706f7f445abeddf7eb2f8e927a => generated 260 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:37.801380 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:37.805780 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb114cce-63de-4f9b-a21f-5ea2c55c7219 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:37.851356 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49699f7-66f4-4768-b9bb-a2096a000543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:37.852059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49699f7-66f4-4768-b9bb-a2096a000543 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:37.852338 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49699f7-66f4-4768-b9bb-a2096a000543 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:37.852612 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49699f7-66f4-4768-b9bb-a2096a000543 None admin] PATH_INFO: `/v3/groups/9bba5ca03ec949a287b22ba2f431d6cc` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:37.853889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49699f7-66f4-4768-b9bb-a2096a000543 None admin] RBAC: Authorizing `identity:get_group(group_id=9bba5ca03ec949a287b22ba2f431d6cc)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:37.855401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49699f7-66f4-4768-b9bb-a2096a000543 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:37.857555 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 189/385] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Sep 5 12:05:37 2022] GET /identity/v3/groups/9bba5ca03ec949a287b22ba2f431d6cc => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:37.863042 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7841dfeb-8050-4c58-885d-e8fa9c62daf7 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:38.007317 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7841dfeb-8050-4c58-885d-e8fa9c62daf7 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:38.054025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a75e44-e66c-4764-a545-3e1f77ca4b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:38.054670 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a75e44-e66c-4764-a545-3e1f77ca4b05 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:38.054953 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a75e44-e66c-4764-a545-3e1f77ca4b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:38.055229 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a75e44-e66c-4764-a545-3e1f77ca4b05 None admin] PATH_INFO: `/v3/projects/405d2cd2fe68456ca034a57f24f6b015` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:38.056456 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a75e44-e66c-4764-a545-3e1f77ca4b05 None admin] RBAC: Authorizing `identity:get_project(project_id=405d2cd2fe68456ca034a57f24f6b015)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:38.057901 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a75e44-e66c-4764-a545-3e1f77ca4b05 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:38.059904 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 197/386] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:37 2022] GET /identity/v3/projects/405d2cd2fe68456ca034a57f24f6b015 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:38.066283 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49699f7-66f4-4768-b9bb-a2096a000543 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:38.189561 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49699f7-66f4-4768-b9bb-a2096a000543 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:38.238880 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abdd05b-820c-44d6-ba30-e558cb7e01e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:38.239572 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abdd05b-820c-44d6-ba30-e558cb7e01e4 None admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:38.239899 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abdd05b-820c-44d6-ba30-e558cb7e01e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:38.240183 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abdd05b-820c-44d6-ba30-e558cb7e01e4 None admin] PATH_INFO: `/v3/projects/405d2cd2fe68456ca034a57f24f6b015/groups/9bba5ca03ec949a287b22ba2f431d6cc/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:38.242128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abdd05b-820c-44d6-ba30-e558cb7e01e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=405d2cd2fe68456ca034a57f24f6b015, group_id=9bba5ca03ec949a287b22ba2f431d6cc, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:38.243621 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abdd05b-820c-44d6-ba30-e558cb7e01e4 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:38.262358 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 190/387] 10.222.0.16 () {58 vars in 1442 bytes} [Mon Sep 5 12:05:38 2022] PUT /identity/v3/projects/405d2cd2fe68456ca034a57f24f6b015/groups/9bba5ca03ec949a287b22ba2f431d6cc/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:05:38.878705 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c54c8-6db3-42cb-bf13-4a9065092f08 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:38.879246 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c54c8-6db3-42cb-bf13-4a9065092f08 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:38.879246 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c54c8-6db3-42cb-bf13-4a9065092f08 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:38.880922 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 198/388] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:38.886820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda58cff-9075-4bf8-a30b-c869afd0d6ac None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:38.887132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda58cff-9075-4bf8-a30b-c869afd0d6ac None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:38.887371 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda58cff-9075-4bf8-a30b-c869afd0d6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:38.921232 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-dda58cff-9075-4bf8-a30b-c869afd0d6ac None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:38.922465 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-dda58cff-9075-4bf8-a30b-c869afd0d6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:38.953574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:39.005243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 191/389] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:05:39.006524 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d2e5b-d9b5-4313-b0e7-3937508ea311 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:39.006814 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d2e5b-d9b5-4313-b0e7-3937508ea311 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:39.007087 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d2e5b-d9b5-4313-b0e7-3937508ea311 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:39.011567 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 199/390] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:39.027674 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92be87-eb4b-4681-991a-f13ff6843d28 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:39.028215 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92be87-eb4b-4681-991a-f13ff6843d28 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:39.028485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92be87-eb4b-4681-991a-f13ff6843d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:39.056616 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-bc92be87-eb4b-4681-991a-f13ff6843d28 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:39.057827 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-bc92be87-eb4b-4681-991a-f13ff6843d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:39.128783 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 192/391] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:39.322706 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d2e5b-d9b5-4313-b0e7-3937508ea311 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:39.358931 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:39.453252 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d2e5b-d9b5-4313-b0e7-3937508ea311 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:39.469110 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc92be87-eb4b-4681-991a-f13ff6843d28 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:39.494879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f9b9e-1bb9-4f5e-9261-f71abb05dfed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:39.495512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f9b9e-1bb9-4f5e-9261-f71abb05dfed None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:39.495823 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f9b9e-1bb9-4f5e-9261-f71abb05dfed None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:39.496333 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f9b9e-1bb9-4f5e-9261-f71abb05dfed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:39.518069 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f9b9e-1bb9-4f5e-9261-f71abb05dfed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:39.523468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-334f9b9e-1bb9-4f5e-9261-f71abb05dfed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:05:39.523468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:39.523468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:39.523468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:39.523468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:39.523468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:39.523468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:39.523468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:39.523468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:39.523468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:39.523468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:39.523468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:39.523468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:39.523468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:39.523468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:39.523468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:39.524379 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:39.524379 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:39.524379 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:39.524379 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:39.524379 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:39.524379 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:39.524379 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:39.524379 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:39.524379 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:39.524379 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:39.524379 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:39.524379 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:39.524379 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:39.524379 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:39.524379 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:39.524379 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:39.525195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:39.525195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:39.525195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:39.525195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:39.525195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:39.525195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:39.525195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:39.525195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:39.525195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:39.525195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:39.525195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:05:39.525195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:39.525195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 200/392] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Sep 5 12:05:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:39.531160 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f9b9e-1bb9-4f5e-9261-f71abb05dfed None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:39.613236 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:39.626974 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc92be87-eb4b-4681-991a-f13ff6843d28 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:39.672961 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:39.677631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94059a-d648-4fcf-8374-8078d192d0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:39.678756 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94059a-d648-4fcf-8374-8078d192d0fc None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:39.679083 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94059a-d648-4fcf-8374-8078d192d0fc None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:39.679656 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94059a-d648-4fcf-8374-8078d192d0fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:39.684424 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f9b9e-1bb9-4f5e-9261-f71abb05dfed None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:39.693675 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94059a-d648-4fcf-8374-8078d192d0fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:39.700676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-0f94059a-d648-4fcf-8374-8078d192d0fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:39.700676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:39.700676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:39.700676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:39.700676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:39.700676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:39.700676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:39.700676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:39.700676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:39.700676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:39.700676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:39.700676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:39.700676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:39.700676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:39.700676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:39.700676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:39.701668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:39.701668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:39.701668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:39.701668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:39.701668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:39.701668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:39.701668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:39.701668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:39.701668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:39.701668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:39.701668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:39.701668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:39.701668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:39.701668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:39.701668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:39.701668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:39.702489 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:39.702489 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:39.702489 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:39.702489 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:39.702489 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:39.702489 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:39.702489 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:39.702489 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:39.702489 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:39.702489 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:39.702489 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:39.702489 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:39.702489 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 193/393] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:39.703690 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a2dbe-67f1-456a-bd9e-3b407ec88746 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:39.703690 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a2dbe-67f1-456a-bd9e-3b407ec88746 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:39.707560 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a2dbe-67f1-456a-bd9e-3b407ec88746 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:39.709649 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 194/394] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:39.710727 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a2dbe-67f1-456a-bd9e-3b407ec88746 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:39.730639 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c36cf6-2982-44f3-bca4-2b4ed1583810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:39.731323 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c36cf6-2982-44f3-bca4-2b4ed1583810 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:39.731639 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c36cf6-2982-44f3-bca4-2b4ed1583810 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:39.731964 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c36cf6-2982-44f3-bca4-2b4ed1583810 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:39.733347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c36cf6-2982-44f3-bca4-2b4ed1583810 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:39.733657 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c36cf6-2982-44f3-bca4-2b4ed1583810 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:39.735139 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c36cf6-2982-44f3-bca4-2b4ed1583810 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:39.744912 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 201/395] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:05:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:39.746375 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a586adf-b27c-4f7c-aca2-d41c56ed4eaf None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:39.746710 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a586adf-b27c-4f7c-aca2-d41c56ed4eaf None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:39.746985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a586adf-b27c-4f7c-aca2-d41c56ed4eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:39.767234 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-4a586adf-b27c-4f7c-aca2-d41c56ed4eaf None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:39.768730 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4a586adf-b27c-4f7c-aca2-d41c56ed4eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:39.847467 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 202/396] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:39.849563 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a586adf-b27c-4f7c-aca2-d41c56ed4eaf None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:39.861650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a2dbe-67f1-456a-bd9e-3b407ec88746 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:39.929737 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ab983-bafe-4c52-85b6-4333b49ff035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:39.930347 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ab983-bafe-4c52-85b6-4333b49ff035 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:39.930806 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ab983-bafe-4c52-85b6-4333b49ff035 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:39.931216 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ab983-bafe-4c52-85b6-4333b49ff035 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:39.932053 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ab983-bafe-4c52-85b6-4333b49ff035 None admin] RBAC: Adding query filter params (name=member) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:39.932513 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ab983-bafe-4c52-85b6-4333b49ff035 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:39.934006 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ab983-bafe-4c52-85b6-4333b49ff035 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:39.942868 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 195/397] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Sep 5 12:05:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:39.948560 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ab983-bafe-4c52-85b6-4333b49ff035 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:39.967970 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:39.979154 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:40.002364 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a586adf-b27c-4f7c-aca2-d41c56ed4eaf None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:40.070789 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17679ae-4b99-402d-b05c-1ee993d62897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:40.071429 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17679ae-4b99-402d-b05c-1ee993d62897 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:40.071741 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17679ae-4b99-402d-b05c-1ee993d62897 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:40.072019 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17679ae-4b99-402d-b05c-1ee993d62897 None admin] PATH_INFO: `/v3/groups/9bba5ca03ec949a287b22ba2f431d6cc` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:40.073210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17679ae-4b99-402d-b05c-1ee993d62897 None admin] RBAC: Authorizing `identity:get_group(group_id=9bba5ca03ec949a287b22ba2f431d6cc)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:40.074641 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17679ae-4b99-402d-b05c-1ee993d62897 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:40.080883 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 203/398] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Sep 5 12:05:39 2022] GET /identity/v3/groups/9bba5ca03ec949a287b22ba2f431d6cc => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:40.084878 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17679ae-4b99-402d-b05c-1ee993d62897 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:40.135993 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ab983-bafe-4c52-85b6-4333b49ff035 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:40.176581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf413d-706e-496f-98f6-67d9a24c0d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:40.177494 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf413d-706e-496f-98f6-67d9a24c0d1a None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:40.177957 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf413d-706e-496f-98f6-67d9a24c0d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:40.178373 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf413d-706e-496f-98f6-67d9a24c0d1a None admin] PATH_INFO: `/v3/groups/9bba5ca03ec949a287b22ba2f431d6cc` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:40.179713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf413d-706e-496f-98f6-67d9a24c0d1a None admin] RBAC: Authorizing `identity:get_group(group_id=9bba5ca03ec949a287b22ba2f431d6cc)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:40.182011 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf413d-706e-496f-98f6-67d9a24c0d1a None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:40.184629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 196/399] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Sep 5 12:05:39 2022] GET /identity/v3/groups/9bba5ca03ec949a287b22ba2f431d6cc => generated 226 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:40.190693 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf413d-706e-496f-98f6-67d9a24c0d1a None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:40.260732 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17679ae-4b99-402d-b05c-1ee993d62897 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:40.290673 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:40.300737 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c275e-3faa-4001-9071-8fad8d5b77c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:40.301448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c275e-3faa-4001-9071-8fad8d5b77c3 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:40.301728 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c275e-3faa-4001-9071-8fad8d5b77c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:40.301998 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c275e-3faa-4001-9071-8fad8d5b77c3 None admin] PATH_INFO: `/v3/projects/405d2cd2fe68456ca034a57f24f6b015` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:40.303110 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5c275e-3faa-4001-9071-8fad8d5b77c3 None admin] RBAC: Authorizing `identity:get_project(project_id=405d2cd2fe68456ca034a57f24f6b015)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:40.304587 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5c275e-3faa-4001-9071-8fad8d5b77c3 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:40.306418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 204/400] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:40 2022] GET /identity/v3/projects/405d2cd2fe68456ca034a57f24f6b015 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:40.312158 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c275e-3faa-4001-9071-8fad8d5b77c3 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:40.314608 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf413d-706e-496f-98f6-67d9a24c0d1a None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:40.333493 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:40.353136 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee6be8-3c51-4ea4-8428-69601dcf3fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:40.353773 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee6be8-3c51-4ea4-8428-69601dcf3fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:40.354053 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee6be8-3c51-4ea4-8428-69601dcf3fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:40.354328 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee6be8-3c51-4ea4-8428-69601dcf3fd6 None admin] PATH_INFO: `/v3/projects/75434cc693884f639e8b84015d47cc58` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:40.355463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee6be8-3c51-4ea4-8428-69601dcf3fd6 None admin] RBAC: Authorizing `identity:get_project(project_id=75434cc693884f639e8b84015d47cc58)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:40.356945 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee6be8-3c51-4ea4-8428-69601dcf3fd6 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:40.358613 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 197/401] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:40 2022] GET /identity/v3/projects/75434cc693884f639e8b84015d47cc58 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:40.364496 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ff828-2ef7-4577-b238-3b5ae5ee1e4e None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:40.364784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ff828-2ef7-4577-b238-3b5ae5ee1e4e None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:40.365063 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ff828-2ef7-4577-b238-3b5ae5ee1e4e None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:40.366468 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 198/402] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:40.371236 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ff828-2ef7-4577-b238-3b5ae5ee1e4e None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:40.424192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c275e-3faa-4001-9071-8fad8d5b77c3 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:40.458827 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa87dc-48f2-485c-abf3-f11dea7e2bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:40.459458 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa87dc-48f2-485c-abf3-f11dea7e2bae None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:40.459763 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa87dc-48f2-485c-abf3-f11dea7e2bae None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:40.460038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa87dc-48f2-485c-abf3-f11dea7e2bae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:40.472038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa87dc-48f2-485c-abf3-f11dea7e2bae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:40.479607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-98fa87dc-48f2-485c-abf3-f11dea7e2bae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:05:40.479607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:40.479607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:40.479607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:40.479607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:40.479607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:40.479607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:40.479607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:40.479607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:40.479607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:40.479607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:40.479607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:40.479607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:40.479607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:40.479607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:40.479607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:40.480557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:40.480557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:40.480557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:40.480557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:40.480557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:40.480557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:40.480557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:40.480557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:40.480557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:40.480557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:40.480557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:40.480557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:40.480557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:40.480557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:40.480557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:40.480557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:40.481442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:40.481442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:40.481442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:40.481442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:40.481442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:40.481442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:40.481442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:40.481442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:40.481442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:40.481442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:40.481442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:05:40.481442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:40.481442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 205/403] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Sep 5 12:05:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:40.482788 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa87dc-48f2-485c-abf3-f11dea7e2bae None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:40.485679 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ff828-2ef7-4577-b238-3b5ae5ee1e4e None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:40.524086 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccdca2b-cde0-4b96-a7f0-7c9b503d5307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:40.524560 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccdca2b-cde0-4b96-a7f0-7c9b503d5307 None admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:40.524838 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccdca2b-cde0-4b96-a7f0-7c9b503d5307 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:40.525151 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccdca2b-cde0-4b96-a7f0-7c9b503d5307 None admin] PATH_INFO: `/v3/projects/405d2cd2fe68456ca034a57f24f6b015/groups/9bba5ca03ec949a287b22ba2f431d6cc/roles/20bf8c77837d43b590a2bd9a67d01f84` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:40.527263 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccdca2b-cde0-4b96-a7f0-7c9b503d5307 None admin] RBAC: Authorizing `identity:create_grant(project_id=405d2cd2fe68456ca034a57f24f6b015, group_id=9bba5ca03ec949a287b22ba2f431d6cc, role_id=20bf8c77837d43b590a2bd9a67d01f84)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:40.529244 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccdca2b-cde0-4b96-a7f0-7c9b503d5307 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:40.543182 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 199/404] 10.222.0.16 () {58 vars in 1442 bytes} [Mon Sep 5 12:05:40 2022] PUT /identity/v3/projects/405d2cd2fe68456ca034a57f24f6b015/groups/9bba5ca03ec949a287b22ba2f431d6cc/roles/20bf8c77837d43b590a2bd9a67d01f84 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:05:40.544790 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362f74f-e4dc-497c-9b55-2c286ebb2636 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:40.545353 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362f74f-e4dc-497c-9b55-2c286ebb2636 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:40.545888 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362f74f-e4dc-497c-9b55-2c286ebb2636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:40.560249 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:40.568471 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-f362f74f-e4dc-497c-9b55-2c286ebb2636 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:40.570375 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-f362f74f-e4dc-497c-9b55-2c286ebb2636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:40.592017 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:40.624431 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa87dc-48f2-485c-abf3-f11dea7e2bae None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:40.661175 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 200/405] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:40.662132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362f74f-e4dc-497c-9b55-2c286ebb2636 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:40.686764 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb160d-9466-447e-a57c-895593d7c077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:40.688166 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb160d-9466-447e-a57c-895593d7c077 None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:40.688672 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb160d-9466-447e-a57c-895593d7c077 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:40.689598 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb160d-9466-447e-a57c-895593d7c077 None admin] PATH_INFO: `/v3/projects/75434cc693884f639e8b84015d47cc58/groups/9bba5ca03ec949a287b22ba2f431d6cc/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:40.692780 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb160d-9466-447e-a57c-895593d7c077 None admin] RBAC: Authorizing `identity:create_grant(project_id=75434cc693884f639e8b84015d47cc58, group_id=9bba5ca03ec949a287b22ba2f431d6cc, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:40.694991 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb160d-9466-447e-a57c-895593d7c077 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:40.716837 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 206/406] 10.222.0.16 () {58 vars in 1442 bytes} [Mon Sep 5 12:05:40 2022] PUT /identity/v3/projects/75434cc693884f639e8b84015d47cc58/groups/9bba5ca03ec949a287b22ba2f431d6cc/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:05:40.718433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e5af0-6baa-4fb3-a5ef-8df07ddad3f7 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:40.718936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e5af0-6baa-4fb3-a5ef-8df07ddad3f7 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:40.719721 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e5af0-6baa-4fb3-a5ef-8df07ddad3f7 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:40.721462 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 207/407] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:40.727493 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d994c-f026-421e-b1b2-d5161fd7d2ef None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:40.727857 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d994c-f026-421e-b1b2-d5161fd7d2ef None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:40.728078 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d994c-f026-421e-b1b2-d5161fd7d2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:40.752869 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-009d994c-f026-421e-b1b2-d5161fd7d2ef None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:40.754465 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-009d994c-f026-421e-b1b2-d5161fd7d2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:40.801376 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362f74f-e4dc-497c-9b55-2c286ebb2636 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:40.841764 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 208/408] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:05:40.850187 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c66148-41a7-48bb-ab6a-8c424389808b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:40.851148 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c66148-41a7-48bb-ab6a-8c424389808b None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:40.851683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c66148-41a7-48bb-ab6a-8c424389808b None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:40.852156 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c66148-41a7-48bb-ab6a-8c424389808b None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:40.853086 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c66148-41a7-48bb-ab6a-8c424389808b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:40.853586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c66148-41a7-48bb-ab6a-8c424389808b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:40.855213 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c66148-41a7-48bb-ab6a-8c424389808b None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:40.858727 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:40.866625 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 201/409] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:05:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:40.872425 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d994c-f026-421e-b1b2-d5161fd7d2ef None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:40.957043 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:40.991600 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c66148-41a7-48bb-ab6a-8c424389808b None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:40.995544 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d994c-f026-421e-b1b2-d5161fd7d2ef None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:41.035401 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8763101a-d553-454e-a6c5-06ae0495aec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:41.036139 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763101a-d553-454e-a6c5-06ae0495aec0 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:41.036418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763101a-d553-454e-a6c5-06ae0495aec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:41.036686 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763101a-d553-454e-a6c5-06ae0495aec0 None admin] PATH_INFO: `/v3/groups/9bba5ca03ec949a287b22ba2f431d6cc` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:41.037816 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8763101a-d553-454e-a6c5-06ae0495aec0 None admin] RBAC: Authorizing `identity:get_group(group_id=9bba5ca03ec949a287b22ba2f431d6cc)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:41.039260 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8763101a-d553-454e-a6c5-06ae0495aec0 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:41.041724 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 209/410] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Sep 5 12:05:40 2022] GET /identity/v3/groups/9bba5ca03ec949a287b22ba2f431d6cc => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:41.048279 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8763101a-d553-454e-a6c5-06ae0495aec0 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:41.127814 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c66148-41a7-48bb-ab6a-8c424389808b None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:41.161688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8763101a-d553-454e-a6c5-06ae0495aec0 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:41.169952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e4187-93fa-4c9e-9665-696fe5edde4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:41.171085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e4187-93fa-4c9e-9665-696fe5edde4c None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:41.171366 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e4187-93fa-4c9e-9665-696fe5edde4c None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:41.171673 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e4187-93fa-4c9e-9665-696fe5edde4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:41.182900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e4187-93fa-4c9e-9665-696fe5edde4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:41.188403 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-0b5e4187-93fa-4c9e-9665-696fe5edde4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:05:41.188403 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:41.188403 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:41.188403 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:41.188403 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:41.188403 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:41.188403 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:41.188403 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:41.188403 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:41.188403 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:41.188403 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:41.188403 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:41.188403 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:41.188403 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:41.188403 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:41.188403 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:41.189440 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:41.189440 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:41.189440 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:41.189440 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:41.189440 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:41.189440 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:41.189440 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:41.189440 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:41.189440 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:41.189440 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:41.189440 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:41.189440 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:41.189440 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:41.189440 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:41.189440 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:41.189440 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:41.190528 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:41.190528 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:41.190528 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:41.190528 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:41.190528 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:41.190528 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:41.190528 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:41.190528 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:41.190528 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:41.190528 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:41.190528 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:05:41.190528 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:41.190528 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 202/411] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:05:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:05:41.192059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e4187-93fa-4c9e-9665-696fe5edde4c None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:41.204349 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee425072-f872-402f-a51a-4e5cffb10198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:41.205274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee425072-f872-402f-a51a-4e5cffb10198 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:41.205560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee425072-f872-402f-a51a-4e5cffb10198 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:41.205829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee425072-f872-402f-a51a-4e5cffb10198 None admin] PATH_INFO: `/v3/projects/75434cc693884f639e8b84015d47cc58` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:41.207007 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee425072-f872-402f-a51a-4e5cffb10198 None admin] RBAC: Authorizing `identity:get_project(project_id=75434cc693884f639e8b84015d47cc58)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:41.209038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee425072-f872-402f-a51a-4e5cffb10198 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:41.210597 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 210/412] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:41 2022] GET /identity/v3/projects/75434cc693884f639e8b84015d47cc58 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:41.211403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee425072-f872-402f-a51a-4e5cffb10198 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:41.220750 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:41.256462 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:41.303820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e4187-93fa-4c9e-9665-696fe5edde4c None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:41.327296 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee425072-f872-402f-a51a-4e5cffb10198 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:41.340329 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6776e5c7-c67e-4fd0-828f-f5a97c9ef783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:41.341286 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776e5c7-c67e-4fd0-828f-f5a97c9ef783 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:41.342256 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776e5c7-c67e-4fd0-828f-f5a97c9ef783 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:41.342705 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776e5c7-c67e-4fd0-828f-f5a97c9ef783 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:41.354869 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6776e5c7-c67e-4fd0-828f-f5a97c9ef783 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:41.361416 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-6776e5c7-c67e-4fd0-828f-f5a97c9ef783 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:41.361416 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:41.361416 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:41.361416 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:41.361416 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:41.361416 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:41.361416 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:41.361416 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:41.361416 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:41.361416 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:41.361416 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:41.361416 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:41.361416 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:41.361416 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:41.361416 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:41.361416 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:41.362392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:41.362392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:41.362392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:41.362392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:41.362392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:41.362392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:41.362392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:41.362392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:41.362392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:41.362392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:41.362392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:41.362392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:41.362392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:41.362392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:41.362392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:41.362392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:41.363207 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:41.363207 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:41.363207 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:41.363207 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:41.363207 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:41.363207 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:41.363207 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:41.363207 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:41.363207 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:41.363207 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:41.363207 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:41.363207 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:41.365109 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 203/413] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:41.365762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6776e5c7-c67e-4fd0-828f-f5a97c9ef783 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:41.369976 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61a1ee-3fb8-4bc5-b698-9501bf139ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:41.370608 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61a1ee-3fb8-4bc5-b698-9501bf139ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:41.370917 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61a1ee-3fb8-4bc5-b698-9501bf139ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:41.371152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61a1ee-3fb8-4bc5-b698-9501bf139ca8 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:41.372056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e61a1ee-3fb8-4bc5-b698-9501bf139ca8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:41.372366 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e61a1ee-3fb8-4bc5-b698-9501bf139ca8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:41.373835 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e61a1ee-3fb8-4bc5-b698-9501bf139ca8 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:41.387893 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 211/414] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:41.389350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61a1ee-3fb8-4bc5-b698-9501bf139ca8 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:41.465562 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:41.488048 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6776e5c7-c67e-4fd0-828f-f5a97c9ef783 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:41.506510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61a1ee-3fb8-4bc5-b698-9501bf139ca8 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:41.507682 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:41.526973 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c630034-ad5f-4328-8724-35e79ddc659e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:41.528332 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c630034-ad5f-4328-8724-35e79ddc659e None admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:41.528849 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c630034-ad5f-4328-8724-35e79ddc659e None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:41.529959 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c630034-ad5f-4328-8724-35e79ddc659e None admin] PATH_INFO: `/v3/projects/75434cc693884f639e8b84015d47cc58/groups/9bba5ca03ec949a287b22ba2f431d6cc/roles/20bf8c77837d43b590a2bd9a67d01f84` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:41.532415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c630034-ad5f-4328-8724-35e79ddc659e None admin] RBAC: Authorizing `identity:create_grant(project_id=75434cc693884f639e8b84015d47cc58, group_id=9bba5ca03ec949a287b22ba2f431d6cc, role_id=20bf8c77837d43b590a2bd9a67d01f84)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:41.534072 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c630034-ad5f-4328-8724-35e79ddc659e None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:41.547266 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20adc71-3b1a-467c-844b-6f9b842d930b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:41.548195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20adc71-3b1a-467c-844b-6f9b842d930b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:41.548481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20adc71-3b1a-467c-844b-6f9b842d930b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:41.548756 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20adc71-3b1a-467c-844b-6f9b842d930b None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:41.549427 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20adc71-3b1a-467c-844b-6f9b842d930b None admin] RBAC: Adding query filter params (name=member) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:41.549731 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20adc71-3b1a-467c-844b-6f9b842d930b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:41.551110 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20adc71-3b1a-467c-844b-6f9b842d930b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:41.559646 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 204/415] 10.222.0.16 () {58 vars in 1442 bytes} [Mon Sep 5 12:05:41 2022] PUT /identity/v3/projects/75434cc693884f639e8b84015d47cc58/groups/9bba5ca03ec949a287b22ba2f431d6cc/roles/20bf8c77837d43b590a2bd9a67d01f84 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:05:41.560962 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 212/416] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Sep 5 12:05:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:41.561380 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c630034-ad5f-4328-8724-35e79ddc659e None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:41.569517 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20adc71-3b1a-467c-844b-6f9b842d930b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:41.688784 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20adc71-3b1a-467c-844b-6f9b842d930b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:41.723981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:41.733333 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce94da-74f1-4f0f-a7cf-1e1266570cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:41.734264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce94da-74f1-4f0f-a7cf-1e1266570cae None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:41.735810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce94da-74f1-4f0f-a7cf-1e1266570cae None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:41.736282 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce94da-74f1-4f0f-a7cf-1e1266570cae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:41.815335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c630034-ad5f-4328-8724-35e79ddc659e None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:41.837622 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ce94da-74f1-4f0f-a7cf-1e1266570cae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:41.839471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ce94da-74f1-4f0f-a7cf-1e1266570cae None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:41.871630 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7350bd5-6be7-4895-bb64-0aaba9be811e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:41.872282 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7350bd5-6be7-4895-bb64-0aaba9be811e None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:41.872567 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7350bd5-6be7-4895-bb64-0aaba9be811e None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:41.872838 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7350bd5-6be7-4895-bb64-0aaba9be811e None admin] PATH_INFO: `/v3/groups/335a64d918d94be1a91101811973a892` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:41.875782 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7350bd5-6be7-4895-bb64-0aaba9be811e None admin] RBAC: Authorizing `identity:get_group(group_id=335a64d918d94be1a91101811973a892)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:41.877550 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7350bd5-6be7-4895-bb64-0aaba9be811e None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:41.881397 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 205/417] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Sep 5 12:05:41 2022] GET /identity/v3/groups/335a64d918d94be1a91101811973a892 => generated 229 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:41.892560 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7350bd5-6be7-4895-bb64-0aaba9be811e None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:41.973834 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 213/418] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:05:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:05:41.980535 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce94da-74f1-4f0f-a7cf-1e1266570cae None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:42.011235 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:42.032704 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:42.138337 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7350bd5-6be7-4895-bb64-0aaba9be811e None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:42.159190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce94da-74f1-4f0f-a7cf-1e1266570cae None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:42.188803 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2403b4df-855e-468e-bb28-e5c9e2916119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:42.189452 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403b4df-855e-468e-bb28-e5c9e2916119 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:42.189733 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403b4df-855e-468e-bb28-e5c9e2916119 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:42.190004 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403b4df-855e-468e-bb28-e5c9e2916119 None admin] PATH_INFO: `/v3/projects/898d4110c367417eb23c75ef5e13bda3` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:42.196234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2403b4df-855e-468e-bb28-e5c9e2916119 None admin] RBAC: Authorizing `identity:get_project(project_id=898d4110c367417eb23c75ef5e13bda3)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:42.197686 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2403b4df-855e-468e-bb28-e5c9e2916119 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:42.203544 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 206/419] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:41 2022] GET /identity/v3/projects/898d4110c367417eb23c75ef5e13bda3 => generated 344 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:42.208833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2403b4df-855e-468e-bb28-e5c9e2916119 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:42.213501 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85433c2-50da-4b2f-be53-20ac8169824e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:42.214500 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85433c2-50da-4b2f-be53-20ac8169824e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:42.215849 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85433c2-50da-4b2f-be53-20ac8169824e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:42.216327 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85433c2-50da-4b2f-be53-20ac8169824e None admin] PATH_INFO: `/v3/projects/405d2cd2fe68456ca034a57f24f6b015` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:42.218054 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85433c2-50da-4b2f-be53-20ac8169824e None admin] RBAC: Authorizing `identity:get_project(project_id=405d2cd2fe68456ca034a57f24f6b015)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:42.219744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85433c2-50da-4b2f-be53-20ac8169824e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:42.225669 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 214/420] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:41 2022] GET /identity/v3/projects/405d2cd2fe68456ca034a57f24f6b015 => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:42.234963 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85433c2-50da-4b2f-be53-20ac8169824e None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:42.343252 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:42.350407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85433c2-50da-4b2f-be53-20ac8169824e None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:42.380483 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2403b4df-855e-468e-bb28-e5c9e2916119 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:42.403931 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:42.412987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d0832-5cdc-4f78-87cb-8213871ef2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:42.413622 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d0832-5cdc-4f78-87cb-8213871ef2a3 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:42.414054 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d0832-5cdc-4f78-87cb-8213871ef2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:42.414327 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d0832-5cdc-4f78-87cb-8213871ef2a3 None admin] PATH_INFO: `/v3/groups/9bba5ca03ec949a287b22ba2f431d6cc` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:42.416180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14d0832-5cdc-4f78-87cb-8213871ef2a3 None admin] RBAC: Authorizing `identity:get_group(group_id=9bba5ca03ec949a287b22ba2f431d6cc)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:42.417627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14d0832-5cdc-4f78-87cb-8213871ef2a3 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:42.419374 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 215/421] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Sep 5 12:05:42 2022] GET /identity/v3/groups/9bba5ca03ec949a287b22ba2f431d6cc => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:42.426119 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d0832-5cdc-4f78-87cb-8213871ef2a3 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:42.444999 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d4e71-0ea7-48e7-a071-2a322da53817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:42.445661 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d4e71-0ea7-48e7-a071-2a322da53817 None admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:42.445937 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d4e71-0ea7-48e7-a071-2a322da53817 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:42.446199 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d4e71-0ea7-48e7-a071-2a322da53817 None admin] PATH_INFO: `/v3/projects/898d4110c367417eb23c75ef5e13bda3/groups/335a64d918d94be1a91101811973a892/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:42.449895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d4e71-0ea7-48e7-a071-2a322da53817 None admin] RBAC: Authorizing `identity:create_grant(project_id=898d4110c367417eb23c75ef5e13bda3, group_id=335a64d918d94be1a91101811973a892, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:42.451357 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d4e71-0ea7-48e7-a071-2a322da53817 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:42.474735 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 207/422] 10.222.0.16 () {58 vars in 1442 bytes} [Mon Sep 5 12:05:42 2022] PUT /identity/v3/projects/898d4110c367417eb23c75ef5e13bda3/groups/335a64d918d94be1a91101811973a892/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:05:42.578540 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d0832-5cdc-4f78-87cb-8213871ef2a3 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:42.629701 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35055f99-243e-4772-869f-a19e063c66fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:42.630314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35055f99-243e-4772-869f-a19e063c66fd None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:42.630590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35055f99-243e-4772-869f-a19e063c66fd None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:42.630857 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35055f99-243e-4772-869f-a19e063c66fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:42.631611 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35055f99-243e-4772-869f-a19e063c66fd None admin] RBAC: Adding query filter params (group.id=9bba5ca03ec949a287b22ba2f431d6cc, role.id=ede2986592354e9888dde3123b2d8c82, scope.project.id=405d2cd2fe68456ca034a57f24f6b015) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:42.631919 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35055f99-243e-4772-869f-a19e063c66fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:42.633437 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35055f99-243e-4772-869f-a19e063c66fd None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:42.643857 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 216/423] 10.222.0.16 () {56 vars in 1376 bytes} [Mon Sep 5 12:05:42 2022] GET /identity/v3/role_assignments?group.id=9bba5ca03ec949a287b22ba2f431d6cc&scope.project.id=405d2cd2fe68456ca034a57f24f6b015&role.id=ede2986592354e9888dde3123b2d8c82 => generated 611 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:42.937001 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c390da-92f3-43f0-81ee-dcf78a3a1406 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:42.937487 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c390da-92f3-43f0-81ee-dcf78a3a1406 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:42.937487 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c390da-92f3-43f0-81ee-dcf78a3a1406 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:42.939007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 208/424] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:42.944661 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c6792-73d2-40f8-8dce-8646bae6d176 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:42.944945 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c6792-73d2-40f8-8dce-8646bae6d176 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:42.945217 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c6792-73d2-40f8-8dce-8646bae6d176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:42.969942 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-424c6792-73d2-40f8-8dce-8646bae6d176 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:42.971160 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-424c6792-73d2-40f8-8dce-8646bae6d176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:42.999804 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259968d-b510-4ec0-aff9-04d758dfabb2 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:43.000138 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259968d-b510-4ec0-aff9-04d758dfabb2 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:43.000367 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259968d-b510-4ec0-aff9-04d758dfabb2 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:43.002037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 209/425] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:43.003972 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:43.008860 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f14eeb-ea01-42ad-83d0-80eaf6b180bb None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:43.009142 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f14eeb-ea01-42ad-83d0-80eaf6b180bb None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:43.009414 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f14eeb-ea01-42ad-83d0-80eaf6b180bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:43.033945 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-20f14eeb-ea01-42ad-83d0-80eaf6b180bb None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:43.035190 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-20f14eeb-ea01-42ad-83d0-80eaf6b180bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:43.058903 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 217/426] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:05:43.135267 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 210/427] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:43.386261 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c6792-73d2-40f8-8dce-8646bae6d176 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:43.428122 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f14eeb-ea01-42ad-83d0-80eaf6b180bb None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:43.508899 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c6792-73d2-40f8-8dce-8646bae6d176 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:43.549888 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f14eeb-ea01-42ad-83d0-80eaf6b180bb None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:43.554144 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:43.556228 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f15dda-cc23-4eb4-96aa-dfe61b13a199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:43.556561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f15dda-cc23-4eb4-96aa-dfe61b13a199 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:43.556726 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f15dda-cc23-4eb4-96aa-dfe61b13a199 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:43.556990 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f15dda-cc23-4eb4-96aa-dfe61b13a199 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:43.572643 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f15dda-cc23-4eb4-96aa-dfe61b13a199 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:43.579692 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-f6f15dda-cc23-4eb4-96aa-dfe61b13a199 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:05:43.579692 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:43.579692 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:43.579692 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:43.579692 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:43.579692 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:43.579692 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:43.579692 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:43.579692 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:43.579692 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:43.579692 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:43.579692 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:43.579692 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:43.579692 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:43.579692 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:43.579692 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:43.580627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:43.580627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:43.580627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:43.580627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:43.580627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:43.580627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:43.580627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:43.580627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:43.580627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:43.580627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:43.580627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:43.580627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:43.580627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:43.580627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:43.580627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:43.580627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:43.581442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:43.581442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:43.581442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:43.581442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:43.581442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:43.581442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:43.581442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:43.581442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:43.581442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:43.581442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:43.581442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:05:43.581442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:43.585466 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 218/428] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Sep 5 12:05:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:43.587987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f15dda-cc23-4eb4-96aa-dfe61b13a199 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:43.601901 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b91dcc-0b16-4072-b554-2d41877f07f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:43.602884 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b91dcc-0b16-4072-b554-2d41877f07f4 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:43.603700 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b91dcc-0b16-4072-b554-2d41877f07f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:43.604187 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b91dcc-0b16-4072-b554-2d41877f07f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:43.617532 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b91dcc-0b16-4072-b554-2d41877f07f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:43.623784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-10b91dcc-0b16-4072-b554-2d41877f07f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:43.623784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:43.623784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:43.623784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:43.623784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:43.623784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:43.623784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:43.623784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:43.623784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:43.623784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:43.623784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:43.623784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:43.623784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:43.623784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:43.623784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:43.623784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:43.624769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:43.624769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:43.624769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:43.624769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:43.624769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:43.624769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:43.624769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:43.624769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:43.624769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:43.624769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:43.624769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:43.624769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:43.624769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:43.624769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:43.624769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:43.624769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:43.625601 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:43.625601 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:43.625601 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:43.625601 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:43.625601 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:43.625601 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:43.625601 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:43.625601 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:43.625601 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:43.625601 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:43.625601 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:05:43.625601 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:43.626245 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:43.626317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 211/429] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:05:43.632396 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b91dcc-0b16-4072-b554-2d41877f07f4 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:43.720171 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f15dda-cc23-4eb4-96aa-dfe61b13a199 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:43.761509 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b91dcc-0b16-4072-b554-2d41877f07f4 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:43.766307 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafcf376-f3e6-4a5c-aa42-24f57d49e539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:43.767216 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafcf376-f3e6-4a5c-aa42-24f57d49e539 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:43.767712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafcf376-f3e6-4a5c-aa42-24f57d49e539 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:43.768205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafcf376-f3e6-4a5c-aa42-24f57d49e539 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:43.769077 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafcf376-f3e6-4a5c-aa42-24f57d49e539 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:43.769561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafcf376-f3e6-4a5c-aa42-24f57d49e539 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:43.771889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafcf376-f3e6-4a5c-aa42-24f57d49e539 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:43.789335 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 219/430] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:05:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:43.792626 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafcf376-f3e6-4a5c-aa42-24f57d49e539 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:43.809510 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9c9c6-66cb-4e8e-aa9e-3d60b0f44ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:43.810229 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9c9c6-66cb-4e8e-aa9e-3d60b0f44ffd None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:43.811353 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9c9c6-66cb-4e8e-aa9e-3d60b0f44ffd None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:43.811922 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9c9c6-66cb-4e8e-aa9e-3d60b0f44ffd None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:43.812725 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d9c9c6-66cb-4e8e-aa9e-3d60b0f44ffd None admin] RBAC: Adding query filter params (name=member) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:43.813141 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d9c9c6-66cb-4e8e-aa9e-3d60b0f44ffd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:43.814674 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d9c9c6-66cb-4e8e-aa9e-3d60b0f44ffd None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:43.829325 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 212/431] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Sep 5 12:05:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:43.830633 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9c9c6-66cb-4e8e-aa9e-3d60b0f44ffd None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:43.864678 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:43.912476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:43.940992 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafcf376-f3e6-4a5c-aa42-24f57d49e539 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:43.957159 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9c9c6-66cb-4e8e-aa9e-3d60b0f44ffd None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:43.978648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250a7bc-48bb-4878-9a07-be5dd5327c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:43.979252 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250a7bc-48bb-4878-9a07-be5dd5327c35 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:43.979550 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250a7bc-48bb-4878-9a07-be5dd5327c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:43.979828 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250a7bc-48bb-4878-9a07-be5dd5327c35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:44.001080 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ffff7-d326-4ebe-8646-5df7ffa70281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:44.001691 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ffff7-d326-4ebe-8646-5df7ffa70281 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:44.001968 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ffff7-d326-4ebe-8646-5df7ffa70281 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:44.002240 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ffff7-d326-4ebe-8646-5df7ffa70281 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:44.055267 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250a7bc-48bb-4878-9a07-be5dd5327c35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:44.056796 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250a7bc-48bb-4878-9a07-be5dd5327c35 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:44.084445 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ffff7-d326-4ebe-8646-5df7ffa70281 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:44.085971 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ffff7-d326-4ebe-8646-5df7ffa70281 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:44.118890 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:44.159432 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:44.182840 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 220/432] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:05:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:05:44.184457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4dbec1-3acd-4d86-9b82-81cba41ed641 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:44.184921 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4dbec1-3acd-4d86-9b82-81cba41ed641 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:44.185335 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4dbec1-3acd-4d86-9b82-81cba41ed641 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:44.186942 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 221/433] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:44.192099 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4dbec1-3acd-4d86-9b82-81cba41ed641 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:44.211373 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 213/434] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:05:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 05 12:05:44.217508 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ef329-4755-4845-a898-073d23209357 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:44.217837 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ef329-4755-4845-a898-073d23209357 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:44.218092 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ef329-4755-4845-a898-073d23209357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:44.238831 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-0f7ef329-4755-4845-a898-073d23209357 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:44.240052 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-0f7ef329-4755-4845-a898-073d23209357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:44.320069 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 214/435] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:44.321415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ef329-4755-4845-a898-073d23209357 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:44.332163 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4dbec1-3acd-4d86-9b82-81cba41ed641 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:44.383214 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd134bdd-6bfa-4404-8efd-c21bd8e6f1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:44.383906 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd134bdd-6bfa-4404-8efd-c21bd8e6f1d4 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:44.384187 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd134bdd-6bfa-4404-8efd-c21bd8e6f1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:44.384460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd134bdd-6bfa-4404-8efd-c21bd8e6f1d4 None admin] PATH_INFO: `/v3/projects/405d2cd2fe68456ca034a57f24f6b015` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:44.386099 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd134bdd-6bfa-4404-8efd-c21bd8e6f1d4 None admin] RBAC: Authorizing `identity:get_project(project_id=405d2cd2fe68456ca034a57f24f6b015)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:44.388006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd134bdd-6bfa-4404-8efd-c21bd8e6f1d4 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:44.391560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 222/436] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:44 2022] GET /identity/v3/projects/405d2cd2fe68456ca034a57f24f6b015 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:44.396360 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd134bdd-6bfa-4404-8efd-c21bd8e6f1d4 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:44.409533 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:44.459120 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:44.470889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ef329-4755-4845-a898-073d23209357 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:44.507469 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1863e8-571a-4603-9bd5-cac2a47987b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:44.508209 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1863e8-571a-4603-9bd5-cac2a47987b7 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:44.508493 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1863e8-571a-4603-9bd5-cac2a47987b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:44.508768 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1863e8-571a-4603-9bd5-cac2a47987b7 None admin] PATH_INFO: `/v3/projects/75434cc693884f639e8b84015d47cc58` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:44.510020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1863e8-571a-4603-9bd5-cac2a47987b7 None admin] RBAC: Authorizing `identity:get_project(project_id=75434cc693884f639e8b84015d47cc58)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:44.511618 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1863e8-571a-4603-9bd5-cac2a47987b7 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:44.518731 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 215/437] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:44 2022] GET /identity/v3/projects/75434cc693884f639e8b84015d47cc58 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:44.519676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1863e8-571a-4603-9bd5-cac2a47987b7 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:44.531630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd134bdd-6bfa-4404-8efd-c21bd8e6f1d4 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:44.577719 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9e893-c0e6-4885-8f17-53eced61f784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:44.579116 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9e893-c0e6-4885-8f17-53eced61f784 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:44.579665 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9e893-c0e6-4885-8f17-53eced61f784 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:44.580138 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9e893-c0e6-4885-8f17-53eced61f784 None admin] PATH_INFO: `/v3/groups/9bba5ca03ec949a287b22ba2f431d6cc` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:44.581545 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9e893-c0e6-4885-8f17-53eced61f784 None admin] RBAC: Authorizing `identity:get_group(group_id=9bba5ca03ec949a287b22ba2f431d6cc)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:44.584008 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9e893-c0e6-4885-8f17-53eced61f784 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:44.586950 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 223/438] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Sep 5 12:05:44 2022] GET /identity/v3/groups/9bba5ca03ec949a287b22ba2f431d6cc => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:44.592681 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9e893-c0e6-4885-8f17-53eced61f784 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:44.637437 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1863e8-571a-4603-9bd5-cac2a47987b7 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:44.664745 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:44.676081 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca0d64-0f74-4a8f-aa33-268925ae00e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:44.676722 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca0d64-0f74-4a8f-aa33-268925ae00e0 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:44.677000 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca0d64-0f74-4a8f-aa33-268925ae00e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:44.677265 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca0d64-0f74-4a8f-aa33-268925ae00e0 None admin] PATH_INFO: `/v3/groups/9bba5ca03ec949a287b22ba2f431d6cc` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:44.678388 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bca0d64-0f74-4a8f-aa33-268925ae00e0 None admin] RBAC: Authorizing `identity:get_group(group_id=9bba5ca03ec949a287b22ba2f431d6cc)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:44.679845 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bca0d64-0f74-4a8f-aa33-268925ae00e0 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:44.681737 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 216/439] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Sep 5 12:05:44 2022] GET /identity/v3/groups/9bba5ca03ec949a287b22ba2f431d6cc => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:44.682510 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca0d64-0f74-4a8f-aa33-268925ae00e0 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:44.714082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9e893-c0e6-4885-8f17-53eced61f784 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:44.731094 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:44.749858 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c35e7-275f-46a0-a6c8-be8a22943836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:44.750490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c35e7-275f-46a0-a6c8-be8a22943836 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:44.750770 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c35e7-275f-46a0-a6c8-be8a22943836 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:44.751044 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c35e7-275f-46a0-a6c8-be8a22943836 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:44.761152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669c35e7-275f-46a0-a6c8-be8a22943836 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:44.767005 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-669c35e7-275f-46a0-a6c8-be8a22943836 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:05:44.767005 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:44.767005 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:44.767005 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:44.767005 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:44.767005 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:44.767005 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:44.767005 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:44.767005 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:44.767005 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:44.767005 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:44.767005 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:44.767005 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:44.767005 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:44.767005 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:44.767005 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:44.768028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:44.768028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:44.768028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:44.768028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:44.768028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:44.768028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:44.768028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:44.768028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:44.768028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:44.768028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:44.768028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:44.768028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:44.768028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:44.768028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:44.768028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:44.768028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:44.769080 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:44.769080 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:44.769080 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:44.769080 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:44.769080 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:44.769080 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:44.769080 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:44.769080 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:44.769080 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:44.769080 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:44.769080 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:05:44.769080 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:44.769080 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 224/440] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Sep 5 12:05:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:05:44.769080 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c35e7-275f-46a0-a6c8-be8a22943836 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:44.791163 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca0d64-0f74-4a8f-aa33-268925ae00e0 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:44.825098 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45913a0-37de-440d-8524-611b56682688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:44.825699 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45913a0-37de-440d-8524-611b56682688 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:44.825989 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45913a0-37de-440d-8524-611b56682688 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:44.826789 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45913a0-37de-440d-8524-611b56682688 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:44.827513 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45913a0-37de-440d-8524-611b56682688 None admin] RBAC: Adding query filter params (group.id=9bba5ca03ec949a287b22ba2f431d6cc, role.id=20bf8c77837d43b590a2bd9a67d01f84, scope.project.id=405d2cd2fe68456ca034a57f24f6b015) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:44.827848 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45913a0-37de-440d-8524-611b56682688 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:44.829268 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45913a0-37de-440d-8524-611b56682688 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:44.835088 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 217/441] 10.222.0.16 () {56 vars in 1376 bytes} [Mon Sep 5 12:05:44 2022] GET /identity/v3/role_assignments?group.id=9bba5ca03ec949a287b22ba2f431d6cc&scope.project.id=405d2cd2fe68456ca034a57f24f6b015&role.id=20bf8c77837d43b590a2bd9a67d01f84 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:44.836256 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d1bab-8e6c-4a86-a3f3-5ba7c72657fe None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:44.836571 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d1bab-8e6c-4a86-a3f3-5ba7c72657fe None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:44.836847 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d1bab-8e6c-4a86-a3f3-5ba7c72657fe None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:44.839668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 218/442] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:44.840939 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d1bab-8e6c-4a86-a3f3-5ba7c72657fe None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:44.877460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c35e7-275f-46a0-a6c8-be8a22943836 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:44.911809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:44.916967 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6288b5d9-0198-479b-9ef8-4844e5d36b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:44.917594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288b5d9-0198-479b-9ef8-4844e5d36b32 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:44.917906 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288b5d9-0198-479b-9ef8-4844e5d36b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:44.918139 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288b5d9-0198-479b-9ef8-4844e5d36b32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:44.918872 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6288b5d9-0198-479b-9ef8-4844e5d36b32 None admin] RBAC: Adding query filter params (group.id=9bba5ca03ec949a287b22ba2f431d6cc, role.id=ede2986592354e9888dde3123b2d8c82, scope.project.id=75434cc693884f639e8b84015d47cc58) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:44.919185 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6288b5d9-0198-479b-9ef8-4844e5d36b32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:44.920766 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6288b5d9-0198-479b-9ef8-4844e5d36b32 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:44.927352 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 225/443] 10.222.0.16 () {56 vars in 1376 bytes} [Mon Sep 5 12:05:44 2022] GET /identity/v3/role_assignments?group.id=9bba5ca03ec949a287b22ba2f431d6cc&scope.project.id=75434cc693884f639e8b84015d47cc58&role.id=ede2986592354e9888dde3123b2d8c82 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:44.928463 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d7584-a053-45ae-b76f-2004b7391a3e None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:44.928748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d7584-a053-45ae-b76f-2004b7391a3e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:44.929019 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d7584-a053-45ae-b76f-2004b7391a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:44.961657 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-a04d7584-a053-45ae-b76f-2004b7391a3e None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:44.963845 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a04d7584-a053-45ae-b76f-2004b7391a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:44.980788 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d1bab-8e6c-4a86-a3f3-5ba7c72657fe None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:45.019802 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:45.039763 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82535085-0eb8-4f22-aaf4-1c948efd47bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:45.041123 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82535085-0eb8-4f22-aaf4-1c948efd47bd None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:45.041502 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82535085-0eb8-4f22-aaf4-1c948efd47bd None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:45.041731 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82535085-0eb8-4f22-aaf4-1c948efd47bd None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:45.042655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82535085-0eb8-4f22-aaf4-1c948efd47bd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:45.042926 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82535085-0eb8-4f22-aaf4-1c948efd47bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:45.044474 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82535085-0eb8-4f22-aaf4-1c948efd47bd None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:45.055351 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 219/444] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:05:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:45.061628 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82535085-0eb8-4f22-aaf4-1c948efd47bd None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:45.072261 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 226/445] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:45.172150 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82535085-0eb8-4f22-aaf4-1c948efd47bd None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:45.192148 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:45.211555 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c1b80-f768-4081-b71d-c11eeb19f9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:45.212210 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c1b80-f768-4081-b71d-c11eeb19f9ac None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:45.212494 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c1b80-f768-4081-b71d-c11eeb19f9ac None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:45.212769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c1b80-f768-4081-b71d-c11eeb19f9ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:45.290406 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c1b80-f768-4081-b71d-c11eeb19f9ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:45.291926 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c1b80-f768-4081-b71d-c11eeb19f9ac None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:45.338793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d7584-a053-45ae-b76f-2004b7391a3e None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:45.403506 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 220/446] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:05:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 05 12:05:45.411333 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c1b80-f768-4081-b71d-c11eeb19f9ac None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:45.432328 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:45.465018 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d7584-a053-45ae-b76f-2004b7391a3e None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:45.499874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c3147-49eb-4d2e-9f76-f485ab5e040c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:45.500502 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c3147-49eb-4d2e-9f76-f485ab5e040c None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:45.500774 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c3147-49eb-4d2e-9f76-f485ab5e040c None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:45.501039 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c3147-49eb-4d2e-9f76-f485ab5e040c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:45.510958 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813c3147-49eb-4d2e-9f76-f485ab5e040c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:45.516145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-813c3147-49eb-4d2e-9f76-f485ab5e040c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:05:45.516145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:45.516145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:45.516145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:45.516145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:45.516145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:45.516145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:45.516145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:45.516145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:45.516145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:45.516145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:45.516145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:45.516145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:45.516145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:45.516145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:45.516145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:45.517165 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:45.517165 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:45.517165 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:45.517165 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:45.517165 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:45.517165 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:45.517165 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:45.517165 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:45.517165 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:45.517165 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:45.517165 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:45.517165 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:45.517165 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:45.517165 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:45.517165 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:45.517165 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:45.517966 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:45.517966 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:45.517966 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:45.517966 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:45.517966 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:45.517966 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:45.517966 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:45.517966 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:45.517966 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:45.517966 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:45.517966 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:05:45.517966 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:45.517966 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 227/447] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:05:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:05:45.518859 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c1b80-f768-4081-b71d-c11eeb19f9ac None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:45.523111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c3147-49eb-4d2e-9f76-f485ab5e040c None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:45.553953 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:45.556225 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8272c900-d26b-481d-bad9-31ea1c19f9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:45.557007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272c900-d26b-481d-bad9-31ea1c19f9fe None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:45.557377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272c900-d26b-481d-bad9-31ea1c19f9fe None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:45.557660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272c900-d26b-481d-bad9-31ea1c19f9fe None admin] PATH_INFO: `/v3/projects/75434cc693884f639e8b84015d47cc58` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:45.558798 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8272c900-d26b-481d-bad9-31ea1c19f9fe None admin] RBAC: Authorizing `identity:get_project(project_id=75434cc693884f639e8b84015d47cc58)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:45.560288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8272c900-d26b-481d-bad9-31ea1c19f9fe None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:45.566836 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 221/448] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:45 2022] GET /identity/v3/projects/75434cc693884f639e8b84015d47cc58 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:45.567918 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8272c900-d26b-481d-bad9-31ea1c19f9fe None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:45.624684 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c3147-49eb-4d2e-9f76-f485ab5e040c None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:45.653283 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:45.660269 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d878dc-7c23-407d-ae13-86e51e57e03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:45.660872 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d878dc-7c23-407d-ae13-86e51e57e03f None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:45.661156 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d878dc-7c23-407d-ae13-86e51e57e03f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:45.661423 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d878dc-7c23-407d-ae13-86e51e57e03f None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:45.662081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d878dc-7c23-407d-ae13-86e51e57e03f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:45.662385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d878dc-7c23-407d-ae13-86e51e57e03f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:45.663792 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d878dc-7c23-407d-ae13-86e51e57e03f None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:45.668858 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8272c900-d26b-481d-bad9-31ea1c19f9fe None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:45.672189 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 228/449] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:45.677765 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d878dc-7c23-407d-ae13-86e51e57e03f None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:45.703551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3b6bc-f743-4f0b-8f3e-137dcbcb492a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:45.704182 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3b6bc-f743-4f0b-8f3e-137dcbcb492a None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:45.704462 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3b6bc-f743-4f0b-8f3e-137dcbcb492a None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:45.704731 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3b6bc-f743-4f0b-8f3e-137dcbcb492a None admin] PATH_INFO: `/v3/groups/9bba5ca03ec949a287b22ba2f431d6cc` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:45.705853 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c3b6bc-f743-4f0b-8f3e-137dcbcb492a None admin] RBAC: Authorizing `identity:get_group(group_id=9bba5ca03ec949a287b22ba2f431d6cc)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:45.707288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c3b6bc-f743-4f0b-8f3e-137dcbcb492a None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:45.711860 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 222/450] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Sep 5 12:05:45 2022] GET /identity/v3/groups/9bba5ca03ec949a287b22ba2f431d6cc => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:45.714629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3b6bc-f743-4f0b-8f3e-137dcbcb492a None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:45.773087 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:45.782743 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d878dc-7c23-407d-ae13-86e51e57e03f None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:45.812571 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3b6bc-f743-4f0b-8f3e-137dcbcb492a None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:45.816894 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27560fb9-75f3-46eb-8b66-d33e12497573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:45.817505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27560fb9-75f3-46eb-8b66-d33e12497573 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:45.817787 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27560fb9-75f3-46eb-8b66-d33e12497573 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:45.818062 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27560fb9-75f3-46eb-8b66-d33e12497573 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:45.846512 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2901a5e-d740-4b51-a5c2-6a093dd03291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:45.847110 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2901a5e-d740-4b51-a5c2-6a093dd03291 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:45.847387 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2901a5e-d740-4b51-a5c2-6a093dd03291 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:45.847694 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2901a5e-d740-4b51-a5c2-6a093dd03291 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:45.848415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2901a5e-d740-4b51-a5c2-6a093dd03291 None admin] RBAC: Adding query filter params (group.id=9bba5ca03ec949a287b22ba2f431d6cc, role.id=20bf8c77837d43b590a2bd9a67d01f84, scope.project.id=75434cc693884f639e8b84015d47cc58) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:45.848720 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2901a5e-d740-4b51-a5c2-6a093dd03291 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:45.850138 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2901a5e-d740-4b51-a5c2-6a093dd03291 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:45.856433 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 223/451] 10.222.0.16 () {56 vars in 1376 bytes} [Mon Sep 5 12:05:45 2022] GET /identity/v3/role_assignments?group.id=9bba5ca03ec949a287b22ba2f431d6cc&scope.project.id=75434cc693884f639e8b84015d47cc58&role.id=20bf8c77837d43b590a2bd9a67d01f84 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:45.889972 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27560fb9-75f3-46eb-8b66-d33e12497573 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:45.891489 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27560fb9-75f3-46eb-8b66-d33e12497573 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:45.988331 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:45.998583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 229/452] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:05:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 05 12:05:46.004487 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2901a5e-d740-4b51-a5c2-6a093dd03291 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:46.024653 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:46.108995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2901a5e-d740-4b51-a5c2-6a093dd03291 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:46.143484 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f47a098-185f-4af2-8602-b13f45a24451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:46.144135 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47a098-185f-4af2-8602-b13f45a24451 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:46.144410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47a098-185f-4af2-8602-b13f45a24451 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:46.144685 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47a098-185f-4af2-8602-b13f45a24451 None admin] PATH_INFO: `/v3/projects/898d4110c367417eb23c75ef5e13bda3` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:46.145815 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f47a098-185f-4af2-8602-b13f45a24451 None admin] RBAC: Authorizing `identity:get_project(project_id=898d4110c367417eb23c75ef5e13bda3)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:46.147235 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f47a098-185f-4af2-8602-b13f45a24451 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:46.148888 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 224/453] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:05:46 2022] GET /identity/v3/projects/898d4110c367417eb23c75ef5e13bda3 => generated 344 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:46.154768 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27560fb9-75f3-46eb-8b66-d33e12497573 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:46.256759 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27560fb9-75f3-46eb-8b66-d33e12497573 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:46.289471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad4601f-12be-4c78-b3be-e322df7aba67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:46.290116 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4601f-12be-4c78-b3be-e322df7aba67 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:46.290392 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4601f-12be-4c78-b3be-e322df7aba67 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:46.290661 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4601f-12be-4c78-b3be-e322df7aba67 None admin] PATH_INFO: `/v3/groups/335a64d918d94be1a91101811973a892` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:46.291772 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad4601f-12be-4c78-b3be-e322df7aba67 None admin] RBAC: Authorizing `identity:get_group(group_id=335a64d918d94be1a91101811973a892)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:46.293205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad4601f-12be-4c78-b3be-e322df7aba67 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:46.295343 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 230/454] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Sep 5 12:05:46 2022] GET /identity/v3/groups/335a64d918d94be1a91101811973a892 => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:46.301143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f47a098-185f-4af2-8602-b13f45a24451 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:46.382628 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:46.397509 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f47a098-185f-4af2-8602-b13f45a24451 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:46.445583 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c143d-c2b4-4404-a1ab-e1126314b2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:46.446327 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c143d-c2b4-4404-a1ab-e1126314b2f2 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:46.446454 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c143d-c2b4-4404-a1ab-e1126314b2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:46.446723 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c143d-c2b4-4404-a1ab-e1126314b2f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:46.447454 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c143d-c2b4-4404-a1ab-e1126314b2f2 None admin] RBAC: Adding query filter params (group.id=335a64d918d94be1a91101811973a892, role.id=618c48706f7f445abeddf7eb2f8e927a, scope.project.id=898d4110c367417eb23c75ef5e13bda3) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:46.447802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c143d-c2b4-4404-a1ab-e1126314b2f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:46.449232 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c143d-c2b4-4404-a1ab-e1126314b2f2 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:46.455315 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 225/455] 10.222.0.16 () {56 vars in 1376 bytes} [Mon Sep 5 12:05:46 2022] GET /identity/v3/role_assignments?group.id=335a64d918d94be1a91101811973a892&scope.project.id=898d4110c367417eb23c75ef5e13bda3&role.id=618c48706f7f445abeddf7eb2f8e927a => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:48.970991 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df2d95-c4ba-4c4f-9a00-597360a30650 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:48.971515 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df2d95-c4ba-4c4f-9a00-597360a30650 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:48.971985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df2d95-c4ba-4c4f-9a00-597360a30650 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:48.974482 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 231/456] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:48.980995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df6593-1c8b-4c37-95f8-bd1b452e3dcf None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:48.981513 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df6593-1c8b-4c37-95f8-bd1b452e3dcf None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:48.981934 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df6593-1c8b-4c37-95f8-bd1b452e3dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:49.018091 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-d3df6593-1c8b-4c37-95f8-bd1b452e3dcf None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:49.020068 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-d3df6593-1c8b-4c37-95f8-bd1b452e3dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:49.137387 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfba8b3-bfc0-4d30-afa9-81ba52f5d6a3 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:49.138017 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfba8b3-bfc0-4d30-afa9-81ba52f5d6a3 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:49.139079 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfba8b3-bfc0-4d30-afa9-81ba52f5d6a3 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:49.141071 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 232/457] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:49.146216 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9f98c-72f1-4a69-9745-0e4279f65ff4 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:49.146760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9f98c-72f1-4a69-9745-0e4279f65ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:49.147232 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9f98c-72f1-4a69-9745-0e4279f65ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:49.156907 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 226/458] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:49.176683 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-08a9f98c-72f1-4a69-9745-0e4279f65ff4 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:49.178233 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-08a9f98c-72f1-4a69-9745-0e4279f65ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:49.283632 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:49.286209 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 233/459] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:05:49.433095 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df6593-1c8b-4c37-95f8-bd1b452e3dcf None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:49.534665 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:49.559064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df6593-1c8b-4c37-95f8-bd1b452e3dcf None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:49.595848 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d535f6a-d098-48dc-a724-249c0483a2a0 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:49.596169 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d535f6a-d098-48dc-a724-249c0483a2a0 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:49.596563 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d535f6a-d098-48dc-a724-249c0483a2a0 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:49.599579 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 234/460] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:49.606177 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0a951-dd90-4cfb-aa80-386121226988 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:49.606686 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0a951-dd90-4cfb-aa80-386121226988 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:49.607962 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0a951-dd90-4cfb-aa80-386121226988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:49.610641 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a365c-9286-4cd4-9ea2-bf6fa79f5c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:49.611550 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a365c-9286-4cd4-9ea2-bf6fa79f5c59 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:49.612029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a365c-9286-4cd4-9ea2-bf6fa79f5c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:49.612497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a365c-9286-4cd4-9ea2-bf6fa79f5c59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:49.623556 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a365c-9286-4cd4-9ea2-bf6fa79f5c59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:49.625327 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a365c-9286-4cd4-9ea2-bf6fa79f5c59 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:49.643568 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-767a365c-9286-4cd4-9ea2-bf6fa79f5c59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:05:49.643568 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:49.643568 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:49.643568 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:49.643568 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:49.643568 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:49.643568 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:49.643568 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:49.643568 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:49.643568 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:49.643568 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:49.643568 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:49.643568 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:05:49.643568 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:05:49.643568 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:05:49.643568 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:05:49.645059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:49.645059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:49.645059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:49.645059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:49.645059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:49.645059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:49.645059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:49.645059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:49.645059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:49.645059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:49.645059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:49.645059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:49.645059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:49.645059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:49.645059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:49.645059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:49.645059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:05:49.646724 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:05:49.646724 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:05:49.646724 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:49.648879 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 227/461] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:05:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:49.649724 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-50b0a951-dd90-4cfb-aa80-386121226988 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:49.650416 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a365c-9286-4cd4-9ea2-bf6fa79f5c59 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:49.651835 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-50b0a951-dd90-4cfb-aa80-386121226988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:49.734650 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 235/462] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:49.735660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b0a951-dd90-4cfb-aa80-386121226988 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:49.779819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a365c-9286-4cd4-9ea2-bf6fa79f5c59 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:49.814489 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:49.821191 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ebd75-2e30-4baf-a79f-6e25245c0bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:49.821834 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ebd75-2e30-4baf-a79f-6e25245c0bac None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:49.822117 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ebd75-2e30-4baf-a79f-6e25245c0bac None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:49.822395 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ebd75-2e30-4baf-a79f-6e25245c0bac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:49.835024 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ebd75-2e30-4baf-a79f-6e25245c0bac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:49.836597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ebd75-2e30-4baf-a79f-6e25245c0bac None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:49.853421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-761ebd75-2e30-4baf-a79f-6e25245c0bac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:05:49.853421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:49.853421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:49.853421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:49.853421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:49.853421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:49.853421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:49.853421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:49.853421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:49.853421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:49.853421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:49.853421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:49.853421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:05:49.853421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:05:49.853421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:05:49.853421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:05:49.854377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:49.854377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:49.854377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:49.854377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:49.854377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:49.854377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:49.854377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:49.854377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:49.854377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:49.854377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:49.854377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:49.854377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:49.854377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:49.854377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:49.854377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:49.854377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:49.854377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:05:49.855330 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:05:49.855330 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:05:49.855330 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:49.855330 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 228/463] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:05:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:49.860895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4094d6f-9c1e-46d6-8687-52563add8a97 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:49.861185 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4094d6f-9c1e-46d6-8687-52563add8a97 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:49.861467 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4094d6f-9c1e-46d6-8687-52563add8a97 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:49.863512 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 229/464] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:49.864368 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4094d6f-9c1e-46d6-8687-52563add8a97 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:49.870762 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b0a951-dd90-4cfb-aa80-386121226988 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:49.901387 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:49.919486 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ac2d4-f420-46f7-9151-5f18a4e58f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:49.921093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ac2d4-f420-46f7-9151-5f18a4e58f49 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:49.921579 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ac2d4-f420-46f7-9151-5f18a4e58f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:49.922228 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ac2d4-f420-46f7-9151-5f18a4e58f49 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:49.923461 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ac2d4-f420-46f7-9151-5f18a4e58f49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:49.923963 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ac2d4-f420-46f7-9151-5f18a4e58f49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:49.925518 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ac2d4-f420-46f7-9151-5f18a4e58f49 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:49.942705 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 236/465] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:49.944370 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3858d1-2ac7-451e-bd90-a323ce8303f8 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:49.944843 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3858d1-2ac7-451e-bd90-a323ce8303f8 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:49.945274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3858d1-2ac7-451e-bd90-a323ce8303f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:49.969296 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-9f3858d1-2ac7-451e-bd90-a323ce8303f8 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:49.971172 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9f3858d1-2ac7-451e-bd90-a323ce8303f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:49.988544 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4094d6f-9c1e-46d6-8687-52563add8a97 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:50.033310 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20ac33e-dc63-415c-8b8b-5dc9c67566d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:50.033960 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ac33e-dc63-415c-8b8b-5dc9c67566d5 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:50.034268 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ac33e-dc63-415c-8b8b-5dc9c67566d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:50.034929 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ac33e-dc63-415c-8b8b-5dc9c67566d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:50.035903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20ac33e-dc63-415c-8b8b-5dc9c67566d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:50.036211 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20ac33e-dc63-415c-8b8b-5dc9c67566d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:50.037872 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20ac33e-dc63-415c-8b8b-5dc9c67566d5 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:50.058650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 230/466] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:50.058650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20ac33e-dc63-415c-8b8b-5dc9c67566d5 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:50.061245 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 237/467] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:50.062112 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3858d1-2ac7-451e-bd90-a323ce8303f8 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:50.137572 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:50.200377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3858d1-2ac7-451e-bd90-a323ce8303f8 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:50.209678 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20ac33e-dc63-415c-8b8b-5dc9c67566d5 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:50.214861 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:50.248223 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d66fa80-8b4d-47ac-86df-a52b3c192e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:50.249424 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66fa80-8b4d-47ac-86df-a52b3c192e8f None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:50.249845 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66fa80-8b4d-47ac-86df-a52b3c192e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:50.250337 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66fa80-8b4d-47ac-86df-a52b3c192e8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:50.253453 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b1832-9462-4965-a3f7-146e0a2a3e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:50.254277 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b1832-9462-4965-a3f7-146e0a2a3e6b None admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:50.254701 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b1832-9462-4965-a3f7-146e0a2a3e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:50.255081 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b1832-9462-4965-a3f7-146e0a2a3e6b None admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:50.255886 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1b1832-9462-4965-a3f7-146e0a2a3e6b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:50.257439 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1b1832-9462-4965-a3f7-146e0a2a3e6b None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:50.262856 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d66fa80-8b4d-47ac-86df-a52b3c192e8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:50.265138 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d66fa80-8b4d-47ac-86df-a52b3c192e8f None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:50.282564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-6d66fa80-8b4d-47ac-86df-a52b3c192e8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:05:50.282564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:50.282564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:50.282564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:50.282564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:50.282564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:50.282564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:50.282564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:50.282564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:50.282564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:50.282564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:50.282564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:50.282564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:05:50.282564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:05:50.282564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:05:50.282564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:05:50.283709 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:50.283709 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:50.283709 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:50.283709 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:50.283709 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:50.283709 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:50.283709 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:50.283709 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:50.283709 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:50.283709 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:50.283709 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:50.283709 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:50.283709 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:50.283709 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:50.283709 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:50.283709 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:50.283709 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:05:50.284568 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:05:50.284568 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:05:50.284568 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:50.286388 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 238/468] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:05:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:50.287327 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d66fa80-8b4d-47ac-86df-a52b3c192e8f None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:50.293798 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 231/469] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Sep 5 12:05:50 2022] POST /identity/v3/users => generated 248 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:05:50.294962 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b1832-9462-4965-a3f7-146e0a2a3e6b None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:50.409205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d66fa80-8b4d-47ac-86df-a52b3c192e8f None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:50.428597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:50.432329 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b1832-9462-4965-a3f7-146e0a2a3e6b None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:50.451235 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff809c5-ff25-464f-bdea-b8f897419f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:50.451982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff809c5-ff25-464f-bdea-b8f897419f1f None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:50.452441 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff809c5-ff25-464f-bdea-b8f897419f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:50.452563 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff809c5-ff25-464f-bdea-b8f897419f1f None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:50.453487 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff809c5-ff25-464f-bdea-b8f897419f1f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:50.454718 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff809c5-ff25-464f-bdea-b8f897419f1f None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:50.479100 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a876408-8391-45bf-87a6-eb7c9b172530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:50.479795 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a876408-8391-45bf-87a6-eb7c9b172530 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:50.480079 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a876408-8391-45bf-87a6-eb7c9b172530 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:50.480287 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 239/470] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Sep 5 12:05:50 2022] POST /identity/v3/users => generated 246 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:05:50.482626 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d815a9-84ef-470a-951a-503e0daf9aab None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:50.482626 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d815a9-84ef-470a-951a-503e0daf9aab None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:50.482803 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d815a9-84ef-470a-951a-503e0daf9aab None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:50.485144 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 240/471] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:50.485298 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a876408-8391-45bf-87a6-eb7c9b172530 None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:50.485373 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d815a9-84ef-470a-951a-503e0daf9aab None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:50.485441 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a876408-8391-45bf-87a6-eb7c9b172530 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:50.487683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a876408-8391-45bf-87a6-eb7c9b172530 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:50.488760 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a876408-8391-45bf-87a6-eb7c9b172530 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:50.514668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 232/472] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:50.515909 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:50.516665 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04753adb-1fa4-4c55-aaed-ce3f587807c7 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:50.517195 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04753adb-1fa4-4c55-aaed-ce3f587807c7 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:50.517646 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04753adb-1fa4-4c55-aaed-ce3f587807c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:50.543338 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-04753adb-1fa4-4c55-aaed-ce3f587807c7 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:50.545266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-04753adb-1fa4-4c55-aaed-ce3f587807c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:50.636468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d815a9-84ef-470a-951a-503e0daf9aab None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:50.637398 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 233/473] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:50.638534 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04753adb-1fa4-4c55-aaed-ce3f587807c7 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:50.709875 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb857ccb-4fbf-4057-aa25-0db59b50b243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:50.710850 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb857ccb-4fbf-4057-aa25-0db59b50b243 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:50.711340 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb857ccb-4fbf-4057-aa25-0db59b50b243 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:50.711864 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb857ccb-4fbf-4057-aa25-0db59b50b243 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:50.730920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb857ccb-4fbf-4057-aa25-0db59b50b243 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:50.733875 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb857ccb-4fbf-4057-aa25-0db59b50b243 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:50.762559 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-cb857ccb-4fbf-4057-aa25-0db59b50b243 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:05:50.762559 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:50.762559 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:50.762559 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:50.762559 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:50.762559 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:50.762559 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:50.762559 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:50.762559 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:50.762559 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:50.762559 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:50.762559 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:50.762559 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:05:50.762559 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:05:50.762559 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:05:50.762559 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:05:50.764467 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:50.764467 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:50.764467 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:50.764467 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:50.764467 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:50.764467 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:50.764467 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:50.764467 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:50.764467 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:50.764467 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:50.764467 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:50.764467 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:50.764467 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:50.764467 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:50.764467 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:50.764467 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:50.764467 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:05:50.766155 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:05:50.766155 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:05:50.766155 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:50.766155 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 241/474] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:05:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:50.770646 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb857ccb-4fbf-4057-aa25-0db59b50b243 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:50.811321 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:50.823640 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04753adb-1fa4-4c55-aaed-ce3f587807c7 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:50.867303 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:50.879823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d5622-8f7c-4d26-be66-3922d571c65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:50.880459 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d5622-8f7c-4d26-be66-3922d571c65e None admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:50.880738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d5622-8f7c-4d26-be66-3922d571c65e None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:50.881009 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d5622-8f7c-4d26-be66-3922d571c65e None admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:50.882327 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d5622-8f7c-4d26-be66-3922d571c65e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:50.884362 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d5622-8f7c-4d26-be66-3922d571c65e None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:50.918256 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 234/475] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Sep 5 12:05:50 2022] POST /identity/v3/users => generated 248 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:05:50.921896 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb857ccb-4fbf-4057-aa25-0db59b50b243 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:50.970006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df90345-49e6-4d68-af2a-026a38336ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:50.970591 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df90345-49e6-4d68-af2a-026a38336ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:50.970879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df90345-49e6-4d68-af2a-026a38336ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:50.971154 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df90345-49e6-4d68-af2a-026a38336ab4 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:50.971872 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df90345-49e6-4d68-af2a-026a38336ab4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:50.972513 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df90345-49e6-4d68-af2a-026a38336ab4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:50.973357 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d5622-8f7c-4d26-be66-3922d571c65e None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:50.973964 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df90345-49e6-4d68-af2a-026a38336ab4 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:50.990491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 242/476] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:50.996563 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df90345-49e6-4d68-af2a-026a38336ab4 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:51.140481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df90345-49e6-4d68-af2a-026a38336ab4 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:51.183237 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:51.190491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c58ab1-f301-4855-abe6-f0bae1eee3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:51.191509 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c58ab1-f301-4855-abe6-f0bae1eee3df None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:51.192035 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c58ab1-f301-4855-abe6-f0bae1eee3df None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:51.192506 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c58ab1-f301-4855-abe6-f0bae1eee3df None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:51.193376 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c58ab1-f301-4855-abe6-f0bae1eee3df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:51.195027 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c58ab1-f301-4855-abe6-f0bae1eee3df None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:51.199076 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d5622-8f7c-4d26-be66-3922d571c65e None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:51.224445 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 243/477] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Sep 5 12:05:50 2022] POST /identity/v3/users => generated 249 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:05:51.262904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a80fc-d4d2-4669-8ad2-c627aa096e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:51.263806 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a80fc-d4d2-4669-8ad2-c627aa096e6d None admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:51.264092 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a80fc-d4d2-4669-8ad2-c627aa096e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:51.264360 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a80fc-d4d2-4669-8ad2-c627aa096e6d None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:51.265085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a80fc-d4d2-4669-8ad2-c627aa096e6d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:51.266626 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a80fc-d4d2-4669-8ad2-c627aa096e6d None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:51.280321 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.sql.core [None req-c14a80fc-d4d2-4669-8ad2-c627aa096e6d None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 05 12:05:51.280321 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 05 12:05:51.280321 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [parameters: {'id': '682c6f48e646429584aeab9e12c40999', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 05 12:05:51.280321 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81541) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 05 12:05:51.297466 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-c14a80fc-d4d2-4669-8ad2-c627aa096e6d None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 05 12:05:51.297466 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:51.297466 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 05 12:05:51.297466 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 05 12:05:51.297466 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 05 12:05:51.297466 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 05 12:05:51.297466 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 05 12:05:51.297466 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application result = self._query(query) Sep 05 12:05:51.297466 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 05 12:05:51.297466 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application conn.query(q) Sep 05 12:05:51.297466 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 05 12:05:51.297466 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 05 12:05:51.297466 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 05 12:05:51.297466 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application result.read() Sep 05 12:05:51.297466 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 05 12:05:51.298417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 05 12:05:51.298417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 05 12:05:51.298417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 05 12:05:51.298417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 05 12:05:51.298417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 05 12:05:51.298417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 05 12:05:51.298417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 05 12:05:51.298417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 05 12:05:51.298417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:51.298417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 05 12:05:51.298417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:51.298417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:51.298417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 05 12:05:51.298417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 05 12:05:51.298417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 05 12:05:51.298417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return ref.to_dict() Sep 05 12:05:51.298417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 05 12:05:51.298417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application next(self.gen) Sep 05 12:05:51.299688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 05 12:05:51.299688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application yield resource Sep 05 12:05:51.299688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 05 12:05:51.299688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application next(self.gen) Sep 05 12:05:51.299688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 05 12:05:51.299688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application self.session.rollback() Sep 05 12:05:51.299688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 05 12:05:51.299688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application self.force_reraise() Sep 05 12:05:51.299688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 05 12:05:51.299688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise self.value Sep 05 12:05:51.299688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 05 12:05:51.299688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 05 12:05:51.299688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 05 12:05:51.299688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application session.commit() Sep 05 12:05:51.299688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 05 12:05:51.299688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 05 12:05:51.299688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 05 12:05:51.300876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application self._prepare_impl() Sep 05 12:05:51.300876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 05 12:05:51.300876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application self.session.flush() Sep 05 12:05:51.300876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 05 12:05:51.300876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application self._flush(objects) Sep 05 12:05:51.300876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 05 12:05:51.300876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 05 12:05:51.300876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 05 12:05:51.300876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application compat.raise_( Sep 05 12:05:51.300876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 05 12:05:51.300876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception Sep 05 12:05:51.300876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 05 12:05:51.300876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application flush_context.execute() Sep 05 12:05:51.300876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 05 12:05:51.300876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rec.execute(self) Sep 05 12:05:51.300876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 05 12:05:51.300876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 05 12:05:51.301774 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 05 12:05:51.301774 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 05 12:05:51.301774 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 05 12:05:51.301774 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 05 12:05:51.301774 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 05 12:05:51.301774 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 05 12:05:51.301774 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 05 12:05:51.301774 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 05 12:05:51.301774 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 05 12:05:51.301774 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 05 12:05:51.301774 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 05 12:05:51.301774 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 05 12:05:51.301774 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 05 12:05:51.301774 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 05 12:05:51.301774 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 05 12:05:51.301774 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception Sep 05 12:05:51.302650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 05 12:05:51.302650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 05 12:05:51.302650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 05 12:05:51.302650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 05 12:05:51.302650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 05 12:05:51.302650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application result = self._query(query) Sep 05 12:05:51.302650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 05 12:05:51.302650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application conn.query(q) Sep 05 12:05:51.302650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 05 12:05:51.302650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 05 12:05:51.302650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 05 12:05:51.302650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application result.read() Sep 05 12:05:51.302650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 05 12:05:51.302650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 05 12:05:51.302650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 05 12:05:51.302650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 05 12:05:51.303445 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 05 12:05:51.303445 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 05 12:05:51.303445 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 05 12:05:51.303445 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 05 12:05:51.303445 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 05 12:05:51.303445 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 05 12:05:51.303445 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [parameters: {'id': '682c6f48e646429584aeab9e12c40999', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 05 12:05:51.303445 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 05 12:05:51.303445 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:51.303445 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:05:51.303445 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:51.303445 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:51.303445 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:51.303445 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:51.303445 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:51.303445 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:51.304978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:51.304978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:51.304978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:51.304978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:51.304978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:51.304978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:51.304978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 05 12:05:51.304978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 05 12:05:51.304978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:51.304978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:51.304978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 05 12:05:51.304978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 05 12:05:51.304978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 05 12:05:51.304978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 05 12:05:51.304978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 05 12:05:51.304978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:51.305818 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 235/478] 10.222.0.16 () {60 vars in 1131 bytes} [Mon Sep 5 12:05:50 2022] POST /identity/v3/roles => generated 144 bytes in 326 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:05:51.305883 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c58ab1-f301-4855-abe6-f0bae1eee3df None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:51.437626 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c58ab1-f301-4855-abe6-f0bae1eee3df None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:51.472805 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6ea10-e686-4f88-8bac-5af6d4b3333a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:51.473512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6ea10-e686-4f88-8bac-5af6d4b3333a None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:51.473803 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6ea10-e686-4f88-8bac-5af6d4b3333a None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:51.474073 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6ea10-e686-4f88-8bac-5af6d4b3333a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:51.485311 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c6ea10-e686-4f88-8bac-5af6d4b3333a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:51.490599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-b0c6ea10-e686-4f88-8bac-5af6d4b3333a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:05:51.490599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:51.490599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:51.490599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:51.490599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:51.490599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:51.490599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:51.490599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:51.490599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:51.490599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:51.490599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:51.490599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:51.490599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:51.490599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:51.490599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:51.490599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:51.491490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:51.491490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:51.491490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:51.491490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:51.491490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:51.491490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:51.491490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:51.491490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:51.491490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:51.491490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:51.491490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:51.491490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:51.491490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:51.491490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:51.491490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:51.491490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:51.492330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:51.492330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:51.492330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:51.492330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:51.492330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:51.492330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:51.492330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:51.492330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:51.492330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:51.492330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:51.492330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:05:51.492330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:51.492330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 244/479] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Sep 5 12:05:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:51.500157 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a80fc-d4d2-4669-8ad2-c627aa096e6d None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:51.515997 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:51.622030 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a80fc-d4d2-4669-8ad2-c627aa096e6d None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:51.660873 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a29e5-e25b-407c-93b0-739c1f695285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:51.662264 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a29e5-e25b-407c-93b0-739c1f695285 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:51.663025 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a29e5-e25b-407c-93b0-739c1f695285 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:51.663790 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a29e5-e25b-407c-93b0-739c1f695285 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:51.665240 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a29e5-e25b-407c-93b0-739c1f695285 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:51.666014 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a29e5-e25b-407c-93b0-739c1f695285 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:51.668866 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a29e5-e25b-407c-93b0-739c1f695285 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:51.680980 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 236/480] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:05:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:52.530729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1951af-7f26-4416-a410-162129fdc6a3 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:52.531233 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1951af-7f26-4416-a410-162129fdc6a3 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:52.531337 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1951af-7f26-4416-a410-162129fdc6a3 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:52.532891 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 245/481] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:52.538539 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a4411-f64a-4669-89be-a6e6dae9e0e2 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:52.538980 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a4411-f64a-4669-89be-a6e6dae9e0e2 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:52.539160 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a4411-f64a-4669-89be-a6e6dae9e0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:52.567848 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-001a4411-f64a-4669-89be-a6e6dae9e0e2 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:52.569064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-001a4411-f64a-4669-89be-a6e6dae9e0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:52.671036 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 237/482] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:05:53.050134 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861192b5-2f5f-4f13-ae57-724b790744cb None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:53.050543 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861192b5-2f5f-4f13-ae57-724b790744cb None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:53.050739 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861192b5-2f5f-4f13-ae57-724b790744cb None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:53.052361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 246/483] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:53.061035 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d275a3-a6cf-4488-82ec-b305b4207295 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:53.061326 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d275a3-a6cf-4488-82ec-b305b4207295 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:53.061585 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d275a3-a6cf-4488-82ec-b305b4207295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:53.078308 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:53.092390 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861192b5-2f5f-4f13-ae57-724b790744cb None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:53.100678 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-06d275a3-a6cf-4488-82ec-b305b4207295 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:53.102070 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-06d275a3-a6cf-4488-82ec-b305b4207295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:53.178700 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:53.230334 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 238/484] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:05:53.275835 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861192b5-2f5f-4f13-ae57-724b790744cb None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:53.323706 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a457958-4f2a-405c-a2b8-f5507931a64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:53.324564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a457958-4f2a-405c-a2b8-f5507931a64c None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:53.324993 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a457958-4f2a-405c-a2b8-f5507931a64c None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:53.325413 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a457958-4f2a-405c-a2b8-f5507931a64c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:53.339915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4054d-db0c-4313-8bfc-276d38f6a35c None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:53.340465 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4054d-db0c-4313-8bfc-276d38f6a35c None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:53.340885 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4054d-db0c-4313-8bfc-276d38f6a35c None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:53.342902 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a457958-4f2a-405c-a2b8-f5507931a64c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:53.343199 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 239/485] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:05:53.350656 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-2a457958-4f2a-405c-a2b8-f5507931a64c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:05:53.350656 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:53.350656 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:53.350656 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:53.350656 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:53.350656 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:53.350656 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:53.350656 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:53.350656 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:53.350656 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:53.350656 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:53.350656 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:53.350656 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:53.350656 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:53.350656 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:53.350656 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:53.351782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:53.351782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:53.351782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:53.351782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:53.351782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:53.351782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:53.351782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:53.351782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:53.351782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:53.351782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:53.351782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:53.351782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:53.351782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:53.351782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:53.351782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:53.351782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:53.352753 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:53.352753 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:53.352753 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:53.352753 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:53.352753 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:53.352753 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:53.352753 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:53.352753 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:53.352753 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:53.352753 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:53.352753 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:05:53.352753 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:53.352753 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 247/486] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Sep 5 12:05:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:05:53.355002 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40460357-7fa5-4ed2-882b-7c3aee12f960 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:53.355290 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40460357-7fa5-4ed2-882b-7c3aee12f960 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:53.355606 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40460357-7fa5-4ed2-882b-7c3aee12f960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:53.359757 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a457958-4f2a-405c-a2b8-f5507931a64c None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:53.408202 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-40460357-7fa5-4ed2-882b-7c3aee12f960 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:53.409396 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-40460357-7fa5-4ed2-882b-7c3aee12f960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:53.578609 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a457958-4f2a-405c-a2b8-f5507931a64c None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:53.582001 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 240/487] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:53.590931 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:53.629944 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40460357-7fa5-4ed2-882b-7c3aee12f960 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:53.641029 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3148ee95-7900-4e22-bc04-a2d6e69910ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:53.641691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148ee95-7900-4e22-bc04-a2d6e69910ec None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:53.642127 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148ee95-7900-4e22-bc04-a2d6e69910ec None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:53.642405 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148ee95-7900-4e22-bc04-a2d6e69910ec None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:53.643134 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3148ee95-7900-4e22-bc04-a2d6e69910ec None admin] RBAC: Adding query filter params (name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:53.646357 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3148ee95-7900-4e22-bc04-a2d6e69910ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:53.649027 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3148ee95-7900-4e22-bc04-a2d6e69910ec None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:53.670341 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 248/488] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Sep 5 12:05:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:53.675146 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3148ee95-7900-4e22-bc04-a2d6e69910ec None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:53.756082 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:53.800938 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40460357-7fa5-4ed2-882b-7c3aee12f960 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:53.840363 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12299a25-0007-4d16-b448-ba20c447cca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:53.840922 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12299a25-0007-4d16-b448-ba20c447cca1 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:53.841204 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12299a25-0007-4d16-b448-ba20c447cca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:53.841510 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12299a25-0007-4d16-b448-ba20c447cca1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:53.849273 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3148ee95-7900-4e22-bc04-a2d6e69910ec None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:53.858286 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12299a25-0007-4d16-b448-ba20c447cca1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:53.865776 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-12299a25-0007-4d16-b448-ba20c447cca1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:05:53.865776 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:53.865776 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:53.865776 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:53.865776 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:53.865776 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:53.865776 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:53.865776 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:53.865776 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:53.865776 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:53.865776 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:53.865776 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:53.865776 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:53.865776 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:53.865776 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:53.865776 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:53.866958 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:53.866958 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:53.866958 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:53.866958 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:53.866958 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:53.866958 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:53.866958 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:53.866958 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:53.866958 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:53.866958 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:53.866958 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:53.866958 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:53.866958 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:53.866958 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:53.866958 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:53.866958 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:53.868046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:53.868046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:53.868046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:53.868046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:53.868046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:53.868046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:53.868046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:53.868046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:53.868046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:53.868046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:53.868046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:05:53.868046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:53.870832 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 241/489] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Sep 5 12:05:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:05:53.873301 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a5764-0cc7-4957-9d02-a755097e2205 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:53.873739 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a5764-0cc7-4957-9d02-a755097e2205 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:53.874155 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a5764-0cc7-4957-9d02-a755097e2205 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:53.875965 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 242/490] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:53.877532 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a5764-0cc7-4957-9d02-a755097e2205 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:53.893996 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ea518-3600-4c02-ba49-9bdbc308ad75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:53.894840 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ea518-3600-4c02-ba49-9bdbc308ad75 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:53.895491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ea518-3600-4c02-ba49-9bdbc308ad75 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:53.895762 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ea518-3600-4c02-ba49-9bdbc308ad75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:53.908555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22ea518-3600-4c02-ba49-9bdbc308ad75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:53.910269 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22ea518-3600-4c02-ba49-9bdbc308ad75 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:53.926056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-e22ea518-3600-4c02-ba49-9bdbc308ad75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:05:53.926056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:53.926056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:53.926056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:53.926056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:53.926056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:53.926056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:53.926056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:53.926056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:53.926056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:53.926056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:53.926056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:53.926056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:05:53.926056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:05:53.926056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:05:53.926056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:05:53.927046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:53.927046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:53.927046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:53.927046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:53.927046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:53.927046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:53.927046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:53.927046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:53.927046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:53.927046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:53.927046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:53.927046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:53.927046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:53.927046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:53.927046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:53.927046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:53.927046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:05:53.927961 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:05:53.927961 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:05:53.927961 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:53.927961 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 249/491] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:05:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:05:53.928788 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0a08e-f7eb-46fd-aabf-6c91d4ab7afc None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:53.928788 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0a08e-f7eb-46fd-aabf-6c91d4ab7afc None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:53.928946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0a08e-f7eb-46fd-aabf-6c91d4ab7afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:53.949326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-cfc0a08e-f7eb-46fd-aabf-6c91d4ab7afc None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:53.950556 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-cfc0a08e-f7eb-46fd-aabf-6c91d4ab7afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:53.973995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:54.009659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a5764-0cc7-4957-9d02-a755097e2205 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:54.037624 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:54.048093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 250/492] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:54.048709 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0a08e-f7eb-46fd-aabf-6c91d4ab7afc None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:54.054744 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55d49f-20f1-4616-88ee-b8b8c4012765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:54.055362 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55d49f-20f1-4616-88ee-b8b8c4012765 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:54.055674 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55d49f-20f1-4616-88ee-b8b8c4012765 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:54.055946 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55d49f-20f1-4616-88ee-b8b8c4012765 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:54.056627 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e55d49f-20f1-4616-88ee-b8b8c4012765 None admin] RBAC: Adding query filter params (name=service) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:54.056934 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e55d49f-20f1-4616-88ee-b8b8c4012765 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:54.058394 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e55d49f-20f1-4616-88ee-b8b8c4012765 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:54.073096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 243/493] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Sep 5 12:05:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:54.074104 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55d49f-20f1-4616-88ee-b8b8c4012765 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:54.186930 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0a08e-f7eb-46fd-aabf-6c91d4ab7afc None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:54.208302 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55d49f-20f1-4616-88ee-b8b8c4012765 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:54.222899 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c82a0-f553-4b80-9dcc-bd14a8fa36da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:54.223649 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c82a0-f553-4b80-9dcc-bd14a8fa36da None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:54.223833 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c82a0-f553-4b80-9dcc-bd14a8fa36da None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:54.224095 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c82a0-f553-4b80-9dcc-bd14a8fa36da None admin] PATH_INFO: `/v3/roles/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:54.237558 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c82a0-f553-4b80-9dcc-bd14a8fa36da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:54.243879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-176c82a0-f553-4b80-9dcc-bd14a8fa36da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:05:54.243879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:54.243879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:54.243879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:54.243879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:54.243879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:54.243879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:54.243879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:54.243879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:54.243879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:54.243879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:54.243879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:54.243879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:54.243879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:54.243879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:54.243879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:54.244851 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:54.244851 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:54.244851 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:54.244851 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:54.244851 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:54.244851 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:54.244851 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:54.244851 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:54.244851 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:54.244851 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:54.244851 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:54.244851 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:54.244851 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:54.244851 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:54.244851 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:54.244851 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:54.245670 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:54.245670 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:54.245670 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:54.245670 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:54.245670 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:54.245670 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:54.245670 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:54.245670 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:54.245670 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:54.245670 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:54.245670 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:05:54.245670 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:54.249450 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 251/494] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Sep 5 12:05:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:54.250435 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c82a0-f553-4b80-9dcc-bd14a8fa36da None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:54.262277 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4e2bc-3989-436d-9f78-bd47d4fd013b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:54.263017 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4e2bc-3989-436d-9f78-bd47d4fd013b None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:54.263440 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4e2bc-3989-436d-9f78-bd47d4fd013b None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:54.263835 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4e2bc-3989-436d-9f78-bd47d4fd013b None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:54.264641 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f4e2bc-3989-436d-9f78-bd47d4fd013b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:54.265043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f4e2bc-3989-436d-9f78-bd47d4fd013b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:54.266587 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f4e2bc-3989-436d-9f78-bd47d4fd013b None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:54.267073 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:54.285769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 244/495] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:05:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:54.285769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4e2bc-3989-436d-9f78-bd47d4fd013b None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:54.333564 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:54.384148 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c82a0-f553-4b80-9dcc-bd14a8fa36da None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:54.393164 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4e2bc-3989-436d-9f78-bd47d4fd013b None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:54.421687 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d566bb2a-93a6-4c1d-bc58-d43bb5c1f5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:54.422650 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d566bb2a-93a6-4c1d-bc58-d43bb5c1f5e4 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:54.423120 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d566bb2a-93a6-4c1d-bc58-d43bb5c1f5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:54.423395 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d566bb2a-93a6-4c1d-bc58-d43bb5c1f5e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:54.430170 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16989f9a-631d-493c-a03b-b9f0702e9180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:54.430768 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16989f9a-631d-493c-a03b-b9f0702e9180 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:54.431041 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16989f9a-631d-493c-a03b-b9f0702e9180 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:54.431309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16989f9a-631d-493c-a03b-b9f0702e9180 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:54.432196 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16989f9a-631d-493c-a03b-b9f0702e9180 None admin] RBAC: Adding query filter params (name=service) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:54.432320 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16989f9a-631d-493c-a03b-b9f0702e9180 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:54.433631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16989f9a-631d-493c-a03b-b9f0702e9180 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:54.434294 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d566bb2a-93a6-4c1d-bc58-d43bb5c1f5e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:54.436435 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d566bb2a-93a6-4c1d-bc58-d43bb5c1f5e4 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:54.442703 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 245/496] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Sep 5 12:05:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:54.443817 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16989f9a-631d-493c-a03b-b9f0702e9180 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:54.456625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-d566bb2a-93a6-4c1d-bc58-d43bb5c1f5e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:05:54.456625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:54.456625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:54.456625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:54.456625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:54.456625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:54.456625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:54.456625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:54.456625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:54.456625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:54.456625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:54.456625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:54.456625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:05:54.456625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:05:54.456625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:05:54.456625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:05:54.457586 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:54.457586 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:54.457586 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:54.457586 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:54.457586 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:54.457586 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:54.457586 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:54.457586 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:54.457586 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:54.457586 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:54.457586 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:54.457586 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:54.457586 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:54.457586 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:54.457586 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:54.457586 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:54.457586 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:05:54.458451 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:05:54.458451 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:05:54.458451 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:54.458451 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 252/497] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:05:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:54.459505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d566bb2a-93a6-4c1d-bc58-d43bb5c1f5e4 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:54.521150 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:54.552548 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16989f9a-631d-493c-a03b-b9f0702e9180 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:54.559456 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:54.560557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d566bb2a-93a6-4c1d-bc58-d43bb5c1f5e4 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:54.586708 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2489081-3d7a-4329-aafe-8312b54ed167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:54.587333 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2489081-3d7a-4329-aafe-8312b54ed167 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:54.587638 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2489081-3d7a-4329-aafe-8312b54ed167 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:54.587910 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2489081-3d7a-4329-aafe-8312b54ed167 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:54.595305 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f747d-9127-4345-8ef8-1e6b040bae2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:54.595783 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2489081-3d7a-4329-aafe-8312b54ed167 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:54.595962 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f747d-9127-4345-8ef8-1e6b040bae2a None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:54.596241 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f747d-9127-4345-8ef8-1e6b040bae2a None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:54.596513 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f747d-9127-4345-8ef8-1e6b040bae2a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:54.597056 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2489081-3d7a-4329-aafe-8312b54ed167 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:54.606636 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99f747d-9127-4345-8ef8-1e6b040bae2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:54.610712 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-f2489081-3d7a-4329-aafe-8312b54ed167 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 05 12:05:54.610712 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:54.610712 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:54.610712 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:54.610712 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:54.610712 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:54.610712 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:54.610712 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:54.610712 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:54.610712 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:54.610712 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:54.610712 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:54.610712 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:05:54.610712 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:05:54.610712 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:05:54.610712 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:05:54.611807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:54.611807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:54.611807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:05:54.611807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:05:54.611807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:05:54.611807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:05:54.611807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:54.611807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:54.611807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:54.611807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:54.611807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:54.611807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:54.611807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:54.611807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:54.611807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:54.611807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:54.611807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:54.611807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:54.612756 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:54.612756 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:54.612756 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:05:54.612756 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:05:54.612756 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:05:54.612756 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:05:54.612756 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:05:54.612756 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:05:54.612756 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 05 12:05:54.612756 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:54.613988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-a99f747d-9127-4345-8ef8-1e6b040bae2a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:05:54.613988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:54.613988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:54.613988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:54.613988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:54.613988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:54.613988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:54.613988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:54.613988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:54.613988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:54.613988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:54.613988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:54.613988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:54.613988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:54.613988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:54.613988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:54.615160 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:54.615160 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:54.615160 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:54.615160 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:54.615160 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:54.615160 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:54.615160 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:54.615160 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:54.615160 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:54.615160 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:54.615160 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:54.615160 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:54.615160 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:54.615160 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:54.615160 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:54.615160 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:54.616015 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:54.616015 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:54.616015 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:54.616015 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:54.616015 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:54.616015 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:54.616015 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:54.616015 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:54.616015 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:54.616015 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:54.616015 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:05:54.616015 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:54.617426 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 246/498] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:54.617426 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2489081-3d7a-4329-aafe-8312b54ed167 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:54.617660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 253/499] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Sep 5 12:05:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:54.617660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f747d-9127-4345-8ef8-1e6b040bae2a None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:54.711035 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f747d-9127-4345-8ef8-1e6b040bae2a None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:54.716013 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2489081-3d7a-4329-aafe-8312b54ed167 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:54.734124 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:54.746169 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c825b-3d8b-4faa-bd4e-b4a4faa97093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:54.746780 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c825b-3d8b-4faa-bd4e-b4a4faa97093 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:54.747056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c825b-3d8b-4faa-bd4e-b4a4faa97093 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:54.747327 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c825b-3d8b-4faa-bd4e-b4a4faa97093 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:54.748045 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550c825b-3d8b-4faa-bd4e-b4a4faa97093 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:54.748346 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550c825b-3d8b-4faa-bd4e-b4a4faa97093 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:54.749651 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3dcc9c-d2b2-4c40-8c64-1e691aef6d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:54.749757 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550c825b-3d8b-4faa-bd4e-b4a4faa97093 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:54.750268 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3dcc9c-d2b2-4c40-8c64-1e691aef6d5d None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:54.750544 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3dcc9c-d2b2-4c40-8c64-1e691aef6d5d None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:54.750813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3dcc9c-d2b2-4c40-8c64-1e691aef6d5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:54.761130 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3dcc9c-d2b2-4c40-8c64-1e691aef6d5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:54.761583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 254/500] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:05:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:54.762591 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3dcc9c-d2b2-4c40-8c64-1e691aef6d5d None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:54.762832 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743096d-94d8-4fd6-80fe-a1ef1b6e2597 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:54.763118 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743096d-94d8-4fd6-80fe-a1ef1b6e2597 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:54.763391 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743096d-94d8-4fd6-80fe-a1ef1b6e2597 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:54.764682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 255/501] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:54.765379 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743096d-94d8-4fd6-80fe-a1ef1b6e2597 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:54.781576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-2f3dcc9c-d2b2-4c40-8c64-1e691aef6d5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:05:54.781576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:54.781576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:54.781576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:54.781576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:54.781576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:54.781576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:54.781576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:54.781576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:54.781576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:54.781576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:54.781576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:54.781576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:05:54.781576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:05:54.781576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:05:54.781576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:05:54.782753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:54.782753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:54.782753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:54.782753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:54.782753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:54.782753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:54.782753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:54.782753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:54.782753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:54.782753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:54.782753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:54.782753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:54.782753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:54.782753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:54.782753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:54.782753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:54.782753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:05:54.783637 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:05:54.783637 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:05:54.783637 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:54.783637 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 247/502] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:05:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:54.783637 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3dcc9c-d2b2-4c40-8c64-1e691aef6d5d None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:54.807348 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:54.870328 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743096d-94d8-4fd6-80fe-a1ef1b6e2597 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:54.889192 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3dcc9c-d2b2-4c40-8c64-1e691aef6d5d None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:54.904694 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4dcd3f-1799-4526-9cc9-1182e5a45f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:54.905299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4dcd3f-1799-4526-9cc9-1182e5a45f95 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:54.905581 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4dcd3f-1799-4526-9cc9-1182e5a45f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:54.905859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4dcd3f-1799-4526-9cc9-1182e5a45f95 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:54.906585 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4dcd3f-1799-4526-9cc9-1182e5a45f95 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:54.906895 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4dcd3f-1799-4526-9cc9-1182e5a45f95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:54.908344 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4dcd3f-1799-4526-9cc9-1182e5a45f95 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:54.923322 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b94b7f-56ec-44a5-b59c-9e61f2258f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:54.924024 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b94b7f-56ec-44a5-b59c-9e61f2258f95 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:54.924310 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b94b7f-56ec-44a5-b59c-9e61f2258f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:54.924575 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b94b7f-56ec-44a5-b59c-9e61f2258f95 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:54.925234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b94b7f-56ec-44a5-b59c-9e61f2258f95 None admin] RBAC: Adding query filter params (name=service) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:54.925541 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b94b7f-56ec-44a5-b59c-9e61f2258f95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:54.926900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b94b7f-56ec-44a5-b59c-9e61f2258f95 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:54.935193 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 248/503] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Sep 5 12:05:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:54.935887 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b94b7f-56ec-44a5-b59c-9e61f2258f95 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:54.945192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 256/504] 10.222.0.16 () {56 vars in 1137 bytes} [Mon Sep 5 12:05:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:54.946425 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bb457-69c4-4b0b-b12c-53e34496c6ba None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:54.946717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bb457-69c4-4b0b-b12c-53e34496c6ba None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:54.946988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bb457-69c4-4b0b-b12c-53e34496c6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:54.965270 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-939bb457-69c4-4b0b-b12c-53e34496c6ba None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:54.966388 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-939bb457-69c4-4b0b-b12c-53e34496c6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:55.001614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:55.020435 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:55.035844 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 257/505] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:55.036729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939bb457-69c4-4b0b-b12c-53e34496c6ba None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:55.042283 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b94b7f-56ec-44a5-b59c-9e61f2258f95 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:55.083324 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8683f9-0d85-49e2-a45f-573775db0012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:55.083993 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8683f9-0d85-49e2-a45f-573775db0012 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:55.084822 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8683f9-0d85-49e2-a45f-573775db0012 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:55.084822 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8683f9-0d85-49e2-a45f-573775db0012 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:55.093679 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8683f9-0d85-49e2-a45f-573775db0012 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:55.095128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8683f9-0d85-49e2-a45f-573775db0012 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:55.106548 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-8a8683f9-0d85-49e2-a45f-573775db0012 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:05:55.106548 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:55.106548 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:55.106548 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:55.106548 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:55.106548 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:55.106548 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:55.106548 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:55.106548 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:55.106548 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:55.106548 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:55.106548 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:55.106548 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:05:55.106548 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:05:55.106548 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:05:55.106548 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:05:55.107476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:55.107476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:55.107476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:05:55.107476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:05:55.107476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:05:55.107476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:05:55.107476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:55.107476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:55.107476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:55.107476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:55.107476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:55.107476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:55.107476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:55.107476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:55.107476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:55.107476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:55.107476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:55.107476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:55.108443 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:55.108443 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:55.108443 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:05:55.108443 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:05:55.108443 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:05:55.108443 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:05:55.108443 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:05:55.108443 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:05:55.108443 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:05:55.108443 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:55.108443 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 249/506] 10.222.0.16 () {56 vars in 1093 bytes} [Mon Sep 5 12:05:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:55.108443 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8683f9-0d85-49e2-a45f-573775db0012 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:55.161187 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939bb457-69c4-4b0b-b12c-53e34496c6ba None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:55.196778 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77aee50-b4c5-4647-84a9-a8ecd570607a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:55.197494 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77aee50-b4c5-4647-84a9-a8ecd570607a None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:55.197841 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77aee50-b4c5-4647-84a9-a8ecd570607a None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:55.198186 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77aee50-b4c5-4647-84a9-a8ecd570607a None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:55.198891 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77aee50-b4c5-4647-84a9-a8ecd570607a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:55.199262 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77aee50-b4c5-4647-84a9-a8ecd570607a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:55.201222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77aee50-b4c5-4647-84a9-a8ecd570607a None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:55.212521 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 258/507] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:55.213324 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77aee50-b4c5-4647-84a9-a8ecd570607a None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:55.219007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8683f9-0d85-49e2-a45f-573775db0012 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:55.246986 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:55.259025 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82756ad4-522c-4524-92b0-976f8926504c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:55.259788 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82756ad4-522c-4524-92b0-976f8926504c None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:55.260131 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82756ad4-522c-4524-92b0-976f8926504c None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:55.260472 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82756ad4-522c-4524-92b0-976f8926504c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:55.269884 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:55.272195 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82756ad4-522c-4524-92b0-976f8926504c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:55.273778 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82756ad4-522c-4524-92b0-976f8926504c None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:55.291020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-82756ad4-522c-4524-92b0-976f8926504c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:05:55.291020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:55.291020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:55.291020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:55.291020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:55.291020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:55.291020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:55.291020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:55.291020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:55.291020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:55.291020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:55.291020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:55.291020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:05:55.291020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:05:55.291020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:05:55.291020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:05:55.292203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:55.292203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:55.292203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:55.292203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:55.292203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:55.292203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:55.292203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:55.292203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:55.292203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:55.292203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:55.292203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:55.292203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:55.292203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:55.292203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:55.292203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:55.292203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:55.292203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:05:55.293237 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:05:55.293237 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:05:55.293237 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:55.293237 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 250/508] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:05:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:55.293960 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82756ad4-522c-4524-92b0-976f8926504c None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:55.326601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77aee50-b4c5-4647-84a9-a8ecd570607a None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:55.359317 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16665d7a-d92d-4b09-a331-a08df9586880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:55.359951 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16665d7a-d92d-4b09-a331-a08df9586880 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:55.360229 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16665d7a-d92d-4b09-a331-a08df9586880 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:55.360501 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16665d7a-d92d-4b09-a331-a08df9586880 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:55.404093 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82756ad4-522c-4524-92b0-976f8926504c None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:55.432514 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16665d7a-d92d-4b09-a331-a08df9586880 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:55.433962 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16665d7a-d92d-4b09-a331-a08df9586880 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:55.438809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb4299-a41b-4a7b-b439-58a772a7ec86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:55.439413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb4299-a41b-4a7b-b439-58a772a7ec86 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:55.439721 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb4299-a41b-4a7b-b439-58a772a7ec86 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:55.439996 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb4299-a41b-4a7b-b439-58a772a7ec86 None admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:55.440729 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb4299-a41b-4a7b-b439-58a772a7ec86 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:55.441035 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb4299-a41b-4a7b-b439-58a772a7ec86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:55.442448 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb4299-a41b-4a7b-b439-58a772a7ec86 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:55.463498 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:55.479767 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 251/509] 10.222.0.16 () {56 vars in 1133 bytes} [Mon Sep 5 12:05:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:55.480603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb4299-a41b-4a7b-b439-58a772a7ec86 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:55.532917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:55.534513 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 259/510] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:05:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:05:55.539483 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16665d7a-d92d-4b09-a331-a08df9586880 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:55.583288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb4299-a41b-4a7b-b439-58a772a7ec86 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:55.617373 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d4f72-4788-4491-adb7-d08fed6b7d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:55.618084 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d4f72-4788-4491-adb7-d08fed6b7d08 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:55.618435 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d4f72-4788-4491-adb7-d08fed6b7d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:55.618709 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d4f72-4788-4491-adb7-d08fed6b7d08 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:55.626659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1d4f72-4788-4491-adb7-d08fed6b7d08 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:55.628153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1d4f72-4788-4491-adb7-d08fed6b7d08 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:55.633890 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16665d7a-d92d-4b09-a331-a08df9586880 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:55.640380 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-0a1d4f72-4788-4491-adb7-d08fed6b7d08 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 12:05:55.640380 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:55.640380 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:55.640380 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:55.640380 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:55.640380 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:55.640380 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:55.640380 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:55.640380 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:55.640380 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:55.640380 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:55.640380 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:55.640380 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:05:55.640380 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:05:55.640380 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:05:55.640380 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:05:55.641429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:55.641429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:55.641429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:05:55.641429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:05:55.641429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:05:55.641429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:05:55.641429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:55.641429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:55.641429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:55.641429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:55.641429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:55.641429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:55.641429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:55.641429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:55.641429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:55.641429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:55.641429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:55.641429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:55.647671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:55.647671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:55.647671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:05:55.647671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:05:55.647671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:05:55.647671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:05:55.647671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:05:55.647671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:05:55.647671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 12:05:55.647671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:55.647671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 252/511] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:05:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:55.647671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d4f72-4788-4491-adb7-d08fed6b7d08 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:55.674081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:55.680907 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8e8ca-a648-4088-a5eb-b5d180b0552b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:55.681594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8e8ca-a648-4088-a5eb-b5d180b0552b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:55.681936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8e8ca-a648-4088-a5eb-b5d180b0552b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:55.682277 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8e8ca-a648-4088-a5eb-b5d180b0552b None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:55.682982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8e8ca-a648-4088-a5eb-b5d180b0552b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:55.683355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8e8ca-a648-4088-a5eb-b5d180b0552b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:55.684879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8e8ca-a648-4088-a5eb-b5d180b0552b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:55.696608 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 260/512] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:55.697420 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8e8ca-a648-4088-a5eb-b5d180b0552b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:55.749170 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d4f72-4788-4491-adb7-d08fed6b7d08 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:55.759630 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:55.783775 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a634d-4e67-49b3-91b5-7c3cf6fb44a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:55.784395 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a634d-4e67-49b3-91b5-7c3cf6fb44a9 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:55.784671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a634d-4e67-49b3-91b5-7c3cf6fb44a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:55.784941 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a634d-4e67-49b3-91b5-7c3cf6fb44a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:55.796272 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189a634d-4e67-49b3-91b5-7c3cf6fb44a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:55.797724 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189a634d-4e67-49b3-91b5-7c3cf6fb44a9 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:55.798167 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8e8ca-a648-4088-a5eb-b5d180b0552b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:55.812932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-189a634d-4e67-49b3-91b5-7c3cf6fb44a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:05:55.812932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:55.812932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:55.812932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:55.812932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:55.812932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:55.812932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:55.812932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:55.812932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:55.812932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:55.812932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:55.812932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:55.812932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:05:55.812932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:05:55.812932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:05:55.812932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:05:55.815602 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:55.815602 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:55.815602 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:55.815602 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:55.815602 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:55.815602 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:55.815602 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:55.815602 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:55.815602 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:55.815602 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:55.815602 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:55.815602 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:55.815602 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:55.815602 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:55.815602 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:55.815602 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:55.815602 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:05:55.816560 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:05:55.816560 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:05:55.816560 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:55.816560 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 253/513] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:05:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:55.816560 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a634d-4e67-49b3-91b5-7c3cf6fb44a9 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:55.832907 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af45b6e-a7d5-4052-bc40-428348c0b0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:55.833510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af45b6e-a7d5-4052-bc40-428348c0b0ed None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:55.833795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af45b6e-a7d5-4052-bc40-428348c0b0ed None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:55.834060 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af45b6e-a7d5-4052-bc40-428348c0b0ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:55.893382 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:55.905084 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af45b6e-a7d5-4052-bc40-428348c0b0ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:55.906653 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af45b6e-a7d5-4052-bc40-428348c0b0ed None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:55.918666 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a634d-4e67-49b3-91b5-7c3cf6fb44a9 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:55.966586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed69ec1-7070-4fd8-9301-e14cc5eef8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:55.967213 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed69ec1-7070-4fd8-9301-e14cc5eef8b1 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:55.967489 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed69ec1-7070-4fd8-9301-e14cc5eef8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:55.967796 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed69ec1-7070-4fd8-9301-e14cc5eef8b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:55.978374 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed69ec1-7070-4fd8-9301-e14cc5eef8b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:55.979866 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed69ec1-7070-4fd8-9301-e14cc5eef8b1 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:55.994632 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-9ed69ec1-7070-4fd8-9301-e14cc5eef8b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:05:55.994632 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:55.994632 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:55.994632 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:55.994632 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:55.994632 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:55.994632 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:55.994632 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:55.994632 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:55.994632 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:55.994632 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:55.994632 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:55.994632 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:05:55.994632 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:05:55.994632 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:05:55.994632 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:05:55.995854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:55.995854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:55.995854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:55.995854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:55.995854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:55.995854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:55.995854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:55.995854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:55.995854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:55.995854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:55.995854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:55.995854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:55.995854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:55.995854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:55.995854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:55.995854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:55.995854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:05:55.996718 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:05:55.996718 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:05:55.996718 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:55.996718 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 254/514] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:05:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:55.996718 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed69ec1-7070-4fd8-9301-e14cc5eef8b1 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:56.011127 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 261/515] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:05:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:05:56.012050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af45b6e-a7d5-4052-bc40-428348c0b0ed None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:56.014565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:56.099985 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed69ec1-7070-4fd8-9301-e14cc5eef8b1 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:56.102602 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:56.113233 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af45b6e-a7d5-4052-bc40-428348c0b0ed None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:56.133408 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a8f2a-8af8-4439-839a-cc84a49dadc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:56.134008 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a8f2a-8af8-4439-839a-cc84a49dadc6 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:56.134289 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a8f2a-8af8-4439-839a-cc84a49dadc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:56.134564 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a8f2a-8af8-4439-839a-cc84a49dadc6 None admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:56.135298 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a8f2a-8af8-4439-839a-cc84a49dadc6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:56.135638 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a8f2a-8af8-4439-839a-cc84a49dadc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:56.137064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a8f2a-8af8-4439-839a-cc84a49dadc6 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:56.145925 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196ea06-0f0d-4735-882a-eda83bea1d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:56.146538 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196ea06-0f0d-4735-882a-eda83bea1d61 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:56.146811 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196ea06-0f0d-4735-882a-eda83bea1d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:56.147075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196ea06-0f0d-4735-882a-eda83bea1d61 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:56.154628 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196ea06-0f0d-4735-882a-eda83bea1d61 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:56.154765 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 255/516] 10.222.0.16 () {56 vars in 1137 bytes} [Mon Sep 5 12:05:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:56.155630 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a8f2a-8af8-4439-839a-cc84a49dadc6 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:56.156111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196ea06-0f0d-4735-882a-eda83bea1d61 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:56.171611 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-1196ea06-0f0d-4735-882a-eda83bea1d61 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 05 12:05:56.171611 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:56.171611 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:56.171611 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:56.171611 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:56.171611 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:56.171611 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:56.171611 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:56.171611 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:56.171611 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:56.171611 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:56.171611 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:56.171611 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:05:56.171611 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:05:56.171611 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:05:56.171611 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:05:56.172717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:56.172717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:56.172717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:05:56.172717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:05:56.172717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:05:56.172717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:05:56.172717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:56.172717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:56.172717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:56.172717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:56.172717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:56.172717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:56.172717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:56.172717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:56.172717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:56.172717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:56.172717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:56.172717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:56.173741 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:56.173741 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:56.173741 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:05:56.173741 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:05:56.173741 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:05:56.173741 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:05:56.173741 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:05:56.173741 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:05:56.173741 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 05 12:05:56.173741 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:56.173741 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 262/517] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Sep 5 12:05:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:56.173741 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196ea06-0f0d-4735-882a-eda83bea1d61 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:56.226371 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:56.259247 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a8f2a-8af8-4439-839a-cc84a49dadc6 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:56.276404 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196ea06-0f0d-4735-882a-eda83bea1d61 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:56.296948 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f42aa-e96c-4b43-a1ab-a75d2996b548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:56.297548 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f42aa-e96c-4b43-a1ab-a75d2996b548 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:56.297823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f42aa-e96c-4b43-a1ab-a75d2996b548 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:56.298089 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f42aa-e96c-4b43-a1ab-a75d2996b548 None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:56.298731 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499f42aa-e96c-4b43-a1ab-a75d2996b548 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:56.299029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499f42aa-e96c-4b43-a1ab-a75d2996b548 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:56.300538 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499f42aa-e96c-4b43-a1ab-a75d2996b548 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:56.311948 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 256/518] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:56.312780 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f42aa-e96c-4b43-a1ab-a75d2996b548 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:56.313641 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c2db3-7b14-40f7-9407-6f73801587f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:56.314314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c2db3-7b14-40f7-9407-6f73801587f8 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:56.314679 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c2db3-7b14-40f7-9407-6f73801587f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:56.315020 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c2db3-7b14-40f7-9407-6f73801587f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:56.315757 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028c2db3-7b14-40f7-9407-6f73801587f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:56.317011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028c2db3-7b14-40f7-9407-6f73801587f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:56.317778 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028c2db3-7b14-40f7-9407-6f73801587f8 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:56.329990 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 263/519] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:56.330818 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c2db3-7b14-40f7-9407-6f73801587f8 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:56.343356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:56.421242 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f42aa-e96c-4b43-a1ab-a75d2996b548 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:56.439700 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c2db3-7b14-40f7-9407-6f73801587f8 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:56.452554 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:56.458552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670dae61-e4b1-4ff3-b501-23ee3d3fec66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:56.459181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670dae61-e4b1-4ff3-b501-23ee3d3fec66 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:56.459458 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670dae61-e4b1-4ff3-b501-23ee3d3fec66 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:56.459779 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670dae61-e4b1-4ff3-b501-23ee3d3fec66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:56.471105 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670dae61-e4b1-4ff3-b501-23ee3d3fec66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:56.472590 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670dae61-e4b1-4ff3-b501-23ee3d3fec66 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:56.475873 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569bfba-0c81-483f-a64b-e962367fd4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:56.476481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569bfba-0c81-483f-a64b-e962367fd4df None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:56.476756 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569bfba-0c81-483f-a64b-e962367fd4df None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:56.477026 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569bfba-0c81-483f-a64b-e962367fd4df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:56.486664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-670dae61-e4b1-4ff3-b501-23ee3d3fec66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:05:56.486664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:56.486664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:56.486664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:56.486664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:56.486664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:56.486664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:56.486664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:56.486664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:56.486664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:56.486664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:56.486664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:56.486664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:05:56.486664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:05:56.486664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:05:56.486664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:05:56.487758 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:56.487758 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:56.487758 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:56.487758 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:56.487758 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:56.487758 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:56.487758 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:56.487758 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:56.487758 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:56.487758 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:56.487758 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:56.487758 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:56.487758 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:56.487758 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:56.487758 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:56.487758 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:56.487758 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:05:56.488635 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:05:56.488635 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:05:56.488635 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:56.488635 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 257/520] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:56.488635 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670dae61-e4b1-4ff3-b501-23ee3d3fec66 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:56.553399 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c569bfba-0c81-483f-a64b-e962367fd4df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:56.554939 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c569bfba-0c81-483f-a64b-e962367fd4df None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:56.566153 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:56.598009 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670dae61-e4b1-4ff3-b501-23ee3d3fec66 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:56.632113 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c1e5c-c23d-41c1-b99f-878af89fc145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:56.632766 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c1e5c-c23d-41c1-b99f-878af89fc145 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:56.632982 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c1e5c-c23d-41c1-b99f-878af89fc145 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:56.633254 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c1e5c-c23d-41c1-b99f-878af89fc145 None admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:56.633991 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4c1e5c-c23d-41c1-b99f-878af89fc145 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:56.634291 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4c1e5c-c23d-41c1-b99f-878af89fc145 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:56.635740 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4c1e5c-c23d-41c1-b99f-878af89fc145 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:56.656848 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 258/521] 10.222.0.16 () {56 vars in 1139 bytes} [Mon Sep 5 12:05:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:56.657675 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c1e5c-c23d-41c1-b99f-878af89fc145 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:56.660716 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 264/522] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:05:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:05:56.661668 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569bfba-0c81-483f-a64b-e962367fd4df None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:56.704148 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:56.761954 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c1e5c-c23d-41c1-b99f-878af89fc145 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:56.766177 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569bfba-0c81-483f-a64b-e962367fd4df None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:56.778228 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:56.800942 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2f0ec-187a-422d-952b-b7c0a5a396c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:56.801545 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2f0ec-187a-422d-952b-b7c0a5a396c4 None admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:56.801841 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2f0ec-187a-422d-952b-b7c0a5a396c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:56.802098 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2f0ec-187a-422d-952b-b7c0a5a396c4 None admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:56.802757 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed2f0ec-187a-422d-952b-b7c0a5a396c4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:56.803715 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692cd1d-ded9-4053-8838-43b24a3d6e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:56.804874 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed2f0ec-187a-422d-952b-b7c0a5a396c4 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:56.805013 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692cd1d-ded9-4053-8838-43b24a3d6e29 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:56.805013 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692cd1d-ded9-4053-8838-43b24a3d6e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:56.805013 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692cd1d-ded9-4053-8838-43b24a3d6e29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:56.812513 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9692cd1d-ded9-4053-8838-43b24a3d6e29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:56.813969 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9692cd1d-ded9-4053-8838-43b24a3d6e29 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:56.827036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-9692cd1d-ded9-4053-8838-43b24a3d6e29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:05:56.827036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:56.827036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:56.827036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:56.827036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:56.827036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:56.827036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:56.827036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:56.827036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:56.827036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:56.827036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:56.827036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:56.827036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:05:56.827036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:05:56.827036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:05:56.828231 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:05:56.828231 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:56.828231 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:56.828231 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:56.828231 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:56.828231 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:56.828231 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:56.828231 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:56.828231 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:56.828231 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:56.828231 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:56.828231 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:56.828231 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:56.828231 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:56.828231 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:56.828231 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:56.828231 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:56.829332 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:05:56.829332 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:05:56.829332 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:05:56.829332 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:05:56.829332 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:05:56.829332 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:05:56.829332 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:05:56.829332 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:56.829787 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 259/523] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Sep 5 12:05:56 2022] POST /identity/v3/users => generated 247 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:05:56.829787 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2f0ec-187a-422d-952b-b7c0a5a396c4 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:56.832948 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 265/524] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:05:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:56.833870 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692cd1d-ded9-4053-8838-43b24a3d6e29 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:56.959089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692cd1d-ded9-4053-8838-43b24a3d6e29 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:56.962619 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:56.969328 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2f0ec-187a-422d-952b-b7c0a5a396c4 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:57.000194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500c86e-3f44-4f03-9d1d-dd17087e692a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:57.000798 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500c86e-3f44-4f03-9d1d-dd17087e692a None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:57.001074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500c86e-3f44-4f03-9d1d-dd17087e692a None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:57.001340 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500c86e-3f44-4f03-9d1d-dd17087e692a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:57.022708 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80258b0-b069-4e96-8b00-36d8854e6656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:57.023414 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80258b0-b069-4e96-8b00-36d8854e6656 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:57.023763 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80258b0-b069-4e96-8b00-36d8854e6656 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:57.024036 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80258b0-b069-4e96-8b00-36d8854e6656 None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:57.024778 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80258b0-b069-4e96-8b00-36d8854e6656 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:57.025009 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80258b0-b069-4e96-8b00-36d8854e6656 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:57.026412 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80258b0-b069-4e96-8b00-36d8854e6656 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:57.040530 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 260/525] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:57.041521 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80258b0-b069-4e96-8b00-36d8854e6656 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:57.054511 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:57.088917 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0500c86e-3f44-4f03-9d1d-dd17087e692a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:57.090506 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0500c86e-3f44-4f03-9d1d-dd17087e692a None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:57.148914 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80258b0-b069-4e96-8b00-36d8854e6656 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:57.189099 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea5c1d-421c-4b08-8b4c-8f482671d979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:57.189888 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea5c1d-421c-4b08-8b4c-8f482671d979 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:57.190163 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea5c1d-421c-4b08-8b4c-8f482671d979 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:57.190433 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea5c1d-421c-4b08-8b4c-8f482671d979 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:57.205800 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 266/526] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:05:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:05:57.206024 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea5c1d-421c-4b08-8b4c-8f482671d979 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:57.206142 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500c86e-3f44-4f03-9d1d-dd17087e692a None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:57.206230 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea5c1d-421c-4b08-8b4c-8f482671d979 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:57.222586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-27ea5c1d-421c-4b08-8b4c-8f482671d979 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:05:57.222586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:57.222586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:57.222586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:57.222586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:57.222586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:57.222586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:57.222586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:57.222586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:57.222586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:57.222586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:57.222586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:57.222586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:05:57.222586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:05:57.222586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:05:57.222586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:05:57.224029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:57.224029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:57.224029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:57.224029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:57.224029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:57.224029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:57.224029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:57.224029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:57.224029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:57.224029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:57.224029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:57.224029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:57.224029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:57.224029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:57.224029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:57.224029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:57.224029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:05:57.225316 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:05:57.225316 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:05:57.225316 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:57.225316 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 261/527] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:05:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:57.229002 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea5c1d-421c-4b08-8b4c-8f482671d979 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:57.242439 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:57.287073 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:57.322169 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500c86e-3f44-4f03-9d1d-dd17087e692a None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:57.347775 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea5c1d-421c-4b08-8b4c-8f482671d979 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:57.377184 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31111665-37e1-458f-816a-cb1313100397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:57.377936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31111665-37e1-458f-816a-cb1313100397 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:57.378306 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31111665-37e1-458f-816a-cb1313100397 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:57.378677 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31111665-37e1-458f-816a-cb1313100397 None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:57.379465 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31111665-37e1-458f-816a-cb1313100397 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:57.379954 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31111665-37e1-458f-816a-cb1313100397 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:57.381441 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31111665-37e1-458f-816a-cb1313100397 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:57.385976 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0358172d-d0ec-44f5-a81a-fb2353094c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:57.386606 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358172d-d0ec-44f5-a81a-fb2353094c39 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:57.386883 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358172d-d0ec-44f5-a81a-fb2353094c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:57.387154 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358172d-d0ec-44f5-a81a-fb2353094c39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:57.394330 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0358172d-d0ec-44f5-a81a-fb2353094c39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:57.395816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0358172d-d0ec-44f5-a81a-fb2353094c39 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:57.406189 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 267/528] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:05:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:57.406189 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31111665-37e1-458f-816a-cb1313100397 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:57.408742 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-0358172d-d0ec-44f5-a81a-fb2353094c39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:05:57.408742 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:57.408742 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:57.408742 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:57.408742 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:57.408742 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:57.408742 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:57.408742 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:57.408742 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:57.408742 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:57.408742 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:57.408742 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:57.408742 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:05:57.408742 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:05:57.408742 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:05:57.410551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:05:57.410551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:57.410551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:57.410551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:57.410551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:57.410551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:57.410551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:57.410551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:57.410551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:57.410551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:57.410551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:57.410551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:57.410551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:57.410551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:57.410551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:57.410551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:57.410551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:57.412371 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:05:57.412371 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:05:57.412371 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:05:57.412371 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:05:57.412371 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:05:57.412371 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:05:57.412371 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:05:57.412371 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:57.415778 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 262/529] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:05:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:57.415993 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0358172d-d0ec-44f5-a81a-fb2353094c39 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:57.498789 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:57.530299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31111665-37e1-458f-816a-cb1313100397 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:57.536646 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0358172d-d0ec-44f5-a81a-fb2353094c39 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:57.561987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:57.568075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7bfa87-a1bc-4255-ae9d-c0b676743dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:57.568689 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7bfa87-a1bc-4255-ae9d-c0b676743dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:57.569393 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7bfa87-a1bc-4255-ae9d-c0b676743dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:57.569750 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7bfa87-a1bc-4255-ae9d-c0b676743dc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:57.583815 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7bfa87-a1bc-4255-ae9d-c0b676743dc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:57.585334 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7bfa87-a1bc-4255-ae9d-c0b676743dc9 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:57.587975 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e754b036-3e23-4126-9394-17c0e3106efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:57.588585 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e754b036-3e23-4126-9394-17c0e3106efa None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:57.588862 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e754b036-3e23-4126-9394-17c0e3106efa None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:57.589133 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e754b036-3e23-4126-9394-17c0e3106efa None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:57.589772 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e754b036-3e23-4126-9394-17c0e3106efa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:57.590071 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e754b036-3e23-4126-9394-17c0e3106efa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:57.591962 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e754b036-3e23-4126-9394-17c0e3106efa None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:57.603089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-4c7bfa87-a1bc-4255-ae9d-c0b676743dc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:05:57.603089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:57.603089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:57.603089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:57.603089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:57.603089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:57.603089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:57.603089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:57.603089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:57.603089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:57.603089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:57.603089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:57.603089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:05:57.603089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:05:57.603089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:05:57.603089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:05:57.610167 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:57.610167 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:57.610167 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:57.610167 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:57.610167 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:57.610167 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:57.610167 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:57.610167 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:57.610167 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:57.610167 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:57.610167 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:57.610167 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:57.610167 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:57.610167 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:57.610167 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:57.610167 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:57.610167 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:05:57.611060 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:05:57.611060 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:05:57.611060 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:57.611060 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 268/530] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:05:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:57.611060 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7bfa87-a1bc-4255-ae9d-c0b676743dc9 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:57.611330 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 263/531] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:57.611330 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e754b036-3e23-4126-9394-17c0e3106efa None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:57.730501 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7bfa87-a1bc-4255-ae9d-c0b676743dc9 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:57.734181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e754b036-3e23-4126-9394-17c0e3106efa None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:57.774391 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:57.775993 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09895c05-e664-4220-8593-c96233585774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:57.776690 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09895c05-e664-4220-8593-c96233585774 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:57.777011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09895c05-e664-4220-8593-c96233585774 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:57.777342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09895c05-e664-4220-8593-c96233585774 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:57.778345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09895c05-e664-4220-8593-c96233585774 None admin] RBAC: Adding query filter params (role.id=8cf764a0144a490a822fcf86d7aceeb0, scope.project.id=fa3178684d184ec880f17640d19d55b7, user.id=ed75c0be809d4a0eb0e2ccc59805b1ad) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:57.778736 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09895c05-e664-4220-8593-c96233585774 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:57.780727 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09895c05-e664-4220-8593-c96233585774 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:57.781357 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42454d72-2b00-4d4c-9771-569a88cc03d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:57.782133 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42454d72-2b00-4d4c-9771-569a88cc03d3 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:57.782567 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42454d72-2b00-4d4c-9771-569a88cc03d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:57.783078 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42454d72-2b00-4d4c-9771-569a88cc03d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:57.784090 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42454d72-2b00-4d4c-9771-569a88cc03d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:57.784401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42454d72-2b00-4d4c-9771-569a88cc03d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:57.785856 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42454d72-2b00-4d4c-9771-569a88cc03d3 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:57.795968 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 269/532] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:05:57 2022] GET /identity/v3/role_assignments?user.id=ed75c0be809d4a0eb0e2ccc59805b1ad&scope.project.id=fa3178684d184ec880f17640d19d55b7&role.id=8cf764a0144a490a822fcf86d7aceeb0 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:57.795968 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09895c05-e664-4220-8593-c96233585774 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:57.807228 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 264/533] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:05:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:57.808369 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42454d72-2b00-4d4c-9771-569a88cc03d3 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:57.870355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:57.956943 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09895c05-e664-4220-8593-c96233585774 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:57.965124 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42454d72-2b00-4d4c-9771-569a88cc03d3 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:58.002207 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b9573-b711-469d-b1a3-a00c2430a693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:58.003129 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b9573-b711-469d-b1a3-a00c2430a693 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:58.003576 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b9573-b711-469d-b1a3-a00c2430a693 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:58.003982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b9573-b711-469d-b1a3-a00c2430a693 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:58.013037 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55b9573-b711-469d-b1a3-a00c2430a693 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:58.014676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787013ed-b4ba-4594-a86c-ac0edb7182a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:58.015392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787013ed-b4ba-4594-a86c-ac0edb7182a0 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:58.015683 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55b9573-b711-469d-b1a3-a00c2430a693 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:58.015888 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787013ed-b4ba-4594-a86c-ac0edb7182a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:58.016274 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787013ed-b4ba-4594-a86c-ac0edb7182a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:58.017027 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787013ed-b4ba-4594-a86c-ac0edb7182a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:58.017528 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787013ed-b4ba-4594-a86c-ac0edb7182a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:58.019429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787013ed-b4ba-4594-a86c-ac0edb7182a0 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:58.031471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-a55b9573-b711-469d-b1a3-a00c2430a693 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:05:58.031471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:58.031471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:58.031471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:58.031471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:58.031471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:58.031471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:58.031471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:58.031471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:58.031471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:58.031471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:58.031471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:58.031471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:05:58.031471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:05:58.031471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:05:58.032572 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:05:58.032572 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:58.032572 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:58.032572 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:58.032572 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:58.032572 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:58.032572 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:58.032572 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:58.032572 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:58.032572 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:58.032572 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:58.032572 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:58.032572 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:58.032572 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:58.032572 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:58.032572 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:58.032572 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:58.033617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:05:58.033617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:05:58.033617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:05:58.033617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:05:58.033617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:05:58.033617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:05:58.033617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:05:58.033617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:58.035643 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 265/535] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:58.035806 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 270/535] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:05:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:58.041240 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787013ed-b4ba-4594-a86c-ac0edb7182a0 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:58.041743 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b9573-b711-469d-b1a3-a00c2430a693 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:58.104163 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:58.162508 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787013ed-b4ba-4594-a86c-ac0edb7182a0 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:58.176520 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b9573-b711-469d-b1a3-a00c2430a693 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:58.187282 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:58.206084 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253c549-6be2-4adc-b4e1-14a248027b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:58.207005 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253c549-6be2-4adc-b4e1-14a248027b5a None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:58.207337 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253c549-6be2-4adc-b4e1-14a248027b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:58.207742 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253c549-6be2-4adc-b4e1-14a248027b5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:58.208589 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9253c549-6be2-4adc-b4e1-14a248027b5a None admin] RBAC: Adding query filter params (role.id=8cf764a0144a490a822fcf86d7aceeb0, scope.project.id=fa3178684d184ec880f17640d19d55b7, user.id=7cda290418504f0db85d665479a05353) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:58.209005 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9253c549-6be2-4adc-b4e1-14a248027b5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:58.210674 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9253c549-6be2-4adc-b4e1-14a248027b5a None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:58.217690 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 266/536] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:05:58 2022] GET /identity/v3/role_assignments?user.id=7cda290418504f0db85d665479a05353&scope.project.id=fa3178684d184ec880f17640d19d55b7&role.id=8cf764a0144a490a822fcf86d7aceeb0 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:58.218675 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253c549-6be2-4adc-b4e1-14a248027b5a None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:58.225490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27fddcd-4a62-4d17-85c6-ca7d8aaf0a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:58.226421 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27fddcd-4a62-4d17-85c6-ca7d8aaf0a88 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:58.226915 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27fddcd-4a62-4d17-85c6-ca7d8aaf0a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:58.227331 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27fddcd-4a62-4d17-85c6-ca7d8aaf0a88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:58.237905 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27fddcd-4a62-4d17-85c6-ca7d8aaf0a88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:58.239900 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27fddcd-4a62-4d17-85c6-ca7d8aaf0a88 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:58.254983 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-b27fddcd-4a62-4d17-85c6-ca7d8aaf0a88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:05:58.254983 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:58.254983 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:58.254983 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:58.254983 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:58.254983 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:58.254983 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:58.254983 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:58.254983 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:58.254983 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:58.254983 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:58.254983 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:58.254983 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:05:58.254983 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:05:58.254983 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:05:58.256056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:05:58.256056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:58.256056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:58.256056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:58.256056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:58.256056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:58.256056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:58.256056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:58.256056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:58.256056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:58.256056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:58.256056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:58.256056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:58.256056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:58.256056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:58.256056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:58.256056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:58.257092 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:05:58.257092 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:05:58.257092 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:05:58.257092 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:05:58.257092 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:05:58.257092 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:05:58.257092 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:05:58.257092 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:05:58.259890 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 271/537] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:05:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:58.272621 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27fddcd-4a62-4d17-85c6-ca7d8aaf0a88 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:58.354129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253c549-6be2-4adc-b4e1-14a248027b5a None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:58.375282 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:58.405621 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c3627-d05c-42d8-8d0d-d9aef65952af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:58.406829 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c3627-d05c-42d8-8d0d-d9aef65952af None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:58.408576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c3627-d05c-42d8-8d0d-d9aef65952af None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:58.409290 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c3627-d05c-42d8-8d0d-d9aef65952af None admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:58.411731 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c3627-d05c-42d8-8d0d-d9aef65952af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:58.412353 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c3627-d05c-42d8-8d0d-d9aef65952af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:58.414405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c3627-d05c-42d8-8d0d-d9aef65952af None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:58.443634 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 267/538] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:05:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:58.452947 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c3627-d05c-42d8-8d0d-d9aef65952af None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:58.493223 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27fddcd-4a62-4d17-85c6-ca7d8aaf0a88 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:58.536327 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fcca2e-4b94-455b-8652-b409c38f03e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:58.536729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fcca2e-4b94-455b-8652-b409c38f03e4 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:58.537004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fcca2e-4b94-455b-8652-b409c38f03e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:58.537324 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fcca2e-4b94-455b-8652-b409c38f03e4 None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:58.538069 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fcca2e-4b94-455b-8652-b409c38f03e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:58.538311 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fcca2e-4b94-455b-8652-b409c38f03e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:58.539985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fcca2e-4b94-455b-8652-b409c38f03e4 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:58.553698 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 272/539] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:05:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:58.559855 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fcca2e-4b94-455b-8652-b409c38f03e4 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:58.585210 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c3627-d05c-42d8-8d0d-d9aef65952af None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:58.593495 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:58.626888 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ad280-e39c-4790-9e26-68aff55d1872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:58.628047 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ad280-e39c-4790-9e26-68aff55d1872 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:58.628558 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ad280-e39c-4790-9e26-68aff55d1872 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:58.629134 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ad280-e39c-4790-9e26-68aff55d1872 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:58.630111 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ad280-e39c-4790-9e26-68aff55d1872 None admin] RBAC: Adding query filter params (role.id=8cf764a0144a490a822fcf86d7aceeb0, scope.project.id=fa3178684d184ec880f17640d19d55b7, user.id=e8a47a0d9bfd473cbf0147d7801442d0) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:58.630636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ad280-e39c-4790-9e26-68aff55d1872 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:58.632445 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ad280-e39c-4790-9e26-68aff55d1872 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:58.639706 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 268/540] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:05:58 2022] GET /identity/v3/role_assignments?user.id=e8a47a0d9bfd473cbf0147d7801442d0&scope.project.id=fa3178684d184ec880f17640d19d55b7&role.id=8cf764a0144a490a822fcf86d7aceeb0 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:05:58.708442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fcca2e-4b94-455b-8652-b409c38f03e4 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:58.755082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb0d89-9b33-488e-a18e-a1cbea3dafb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:58.755768 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb0d89-9b33-488e-a18e-a1cbea3dafb3 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:58.756021 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb0d89-9b33-488e-a18e-a1cbea3dafb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:58.756252 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb0d89-9b33-488e-a18e-a1cbea3dafb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:58.757078 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb0d89-9b33-488e-a18e-a1cbea3dafb3 None admin] RBAC: Adding query filter params (role.id=8cf764a0144a490a822fcf86d7aceeb0, scope.project.id=fa3178684d184ec880f17640d19d55b7, user.id=9f79599ed0b74bdbab198627ef72252a) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:05:58.757386 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb0d89-9b33-488e-a18e-a1cbea3dafb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:05:58.758879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb0d89-9b33-488e-a18e-a1cbea3dafb3 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:05:58.765868 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 273/541] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:05:58 2022] GET /identity/v3/role_assignments?user.id=9f79599ed0b74bdbab198627ef72252a&scope.project.id=fa3178684d184ec880f17640d19d55b7&role.id=8cf764a0144a490a822fcf86d7aceeb0 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:05:59.003319 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a468573-4bc8-4f42-adad-dd8bdca667e2 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:59.003830 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a468573-4bc8-4f42-adad-dd8bdca667e2 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:59.003830 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a468573-4bc8-4f42-adad-dd8bdca667e2 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:59.005445 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 269/542] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:05:59.010911 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad99414-32d4-4224-ae9d-9cf1d8e79ca2 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:59.011192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad99414-32d4-4224-ae9d-9cf1d8e79ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:59.011465 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad99414-32d4-4224-ae9d-9cf1d8e79ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:59.030859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-fad99414-32d4-4224-ae9d-9cf1d8e79ca2 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:05:59.032099 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-fad99414-32d4-4224-ae9d-9cf1d8e79ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:05:59.129156 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 274/543] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:05:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:05:59.637901 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a468573-4bc8-4f42-adad-dd8bdca667e2 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:59.688851 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:05:59.760600 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a468573-4bc8-4f42-adad-dd8bdca667e2 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:05:59.800097 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed495f5-4146-44c6-a100-64337f300126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:05:59.801107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed495f5-4146-44c6-a100-64337f300126 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:05:59.801606 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed495f5-4146-44c6-a100-64337f300126 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:05:59.802083 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed495f5-4146-44c6-a100-64337f300126 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:05:59.813650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed495f5-4146-44c6-a100-64337f300126 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:05:59.818805 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-aed495f5-4146-44c6-a100-64337f300126 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:05:59.818805 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:05:59.818805 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:05:59.818805 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:05:59.818805 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:05:59.818805 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:05:59.818805 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:05:59.818805 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:05:59.818805 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:05:59.818805 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:05:59.818805 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:05:59.818805 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:05:59.818805 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:05:59.818805 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:05:59.818805 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:05:59.818805 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:05:59.819713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:05:59.819713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:05:59.819713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:05:59.819713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:05:59.819713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:05:59.819713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:05:59.819713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:05:59.819713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:05:59.819713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:05:59.819713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:05:59.819713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:05:59.819713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:05:59.819713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:05:59.819713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:05:59.819713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:05:59.819713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:05:59.820551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:05:59.820551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:05:59.820551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:05:59.820551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:05:59.820551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:05:59.820551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:05:59.820551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:05:59.820551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:05:59.820551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:05:59.820551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:05:59.820551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:05:59.820551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:05:59.824333 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 270/544] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Sep 5 12:05:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:05:59.832361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad99414-32d4-4224-ae9d-9cf1d8e79ca2 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:05:59.893430 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:00.023814 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad99414-32d4-4224-ae9d-9cf1d8e79ca2 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:00.080243 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b309ef-5bc1-4150-841d-d5eb1e4495a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:00.081257 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b309ef-5bc1-4150-841d-d5eb1e4495a8 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:00.081752 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b309ef-5bc1-4150-841d-d5eb1e4495a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:00.082237 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b309ef-5bc1-4150-841d-d5eb1e4495a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:00.083113 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b309ef-5bc1-4150-841d-d5eb1e4495a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:00.083684 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b309ef-5bc1-4150-841d-d5eb1e4495a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:00.085284 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b309ef-5bc1-4150-841d-d5eb1e4495a8 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:00.097715 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 275/545] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Sep 5 12:05:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:00.104080 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed495f5-4146-44c6-a100-64337f300126 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:00.129805 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02b4bb-493c-4bbe-8759-def9ae9b8463 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:00.130470 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02b4bb-493c-4bbe-8759-def9ae9b8463 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:00.130954 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02b4bb-493c-4bbe-8759-def9ae9b8463 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:00.133418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 276/546] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:00.138779 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a555de0-d477-45fd-8316-f69270ddad9b None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:00.139335 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a555de0-d477-45fd-8316-f69270ddad9b None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:00.139916 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a555de0-d477-45fd-8316-f69270ddad9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:00.172494 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-3a555de0-d477-45fd-8316-f69270ddad9b None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:00.176768 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3a555de0-d477-45fd-8316-f69270ddad9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:00.306074 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:00.311895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed495f5-4146-44c6-a100-64337f300126 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:00.336594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 277/547] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:06:00.405168 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73a332-e1de-42b6-aa90-9ffd9f721b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:00.405834 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73a332-e1de-42b6-aa90-9ffd9f721b55 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:00.406124 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73a332-e1de-42b6-aa90-9ffd9f721b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:00.406392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73a332-e1de-42b6-aa90-9ffd9f721b55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:00.428796 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73a332-e1de-42b6-aa90-9ffd9f721b55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:00.430251 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73a332-e1de-42b6-aa90-9ffd9f721b55 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:00.452995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-0f73a332-e1de-42b6-aa90-9ffd9f721b55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:00.452995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:00.452995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:00.452995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:00.452995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:00.452995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:00.452995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:00.452995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:00.452995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:00.452995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:00.452995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:00.452995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:00.452995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:00.452995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:00.452995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:00.452995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:00.454038 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:00.454038 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:00.454038 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:00.454038 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:00.454038 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:00.454038 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:00.454038 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:00.454038 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:00.454038 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:00.454038 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:00.454038 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:00.454038 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:00.454038 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:00.454038 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:00.454038 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:00.454038 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:00.454038 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:00.456199 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:00.456199 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:00.456199 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:00.456199 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 271/548] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:00.460637 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a555de0-d477-45fd-8316-f69270ddad9b None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:00.499660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:00.595402 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a555de0-d477-45fd-8316-f69270ddad9b None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:00.611909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44949db-c817-4cba-9de8-55818747b272 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:00.612268 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44949db-c817-4cba-9de8-55818747b272 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:00.612439 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44949db-c817-4cba-9de8-55818747b272 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:00.614005 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 272/549] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:06:00.619793 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99999029-9a3a-4402-ac3a-aefd146fbf8f None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:00.620080 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99999029-9a3a-4402-ac3a-aefd146fbf8f None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:00.620333 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99999029-9a3a-4402-ac3a-aefd146fbf8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:00.633445 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73149ae-bb87-40b4-9550-aa309ca4ed2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:00.634065 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73149ae-bb87-40b4-9550-aa309ca4ed2a None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:00.634345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73149ae-bb87-40b4-9550-aa309ca4ed2a None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:00.634619 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73149ae-bb87-40b4-9550-aa309ca4ed2a None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:00.635276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73149ae-bb87-40b4-9550-aa309ca4ed2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:00.635607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73149ae-bb87-40b4-9550-aa309ca4ed2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:00.637012 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73149ae-bb87-40b4-9550-aa309ca4ed2a None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:00.650139 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 278/550] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:00.661918 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73149ae-bb87-40b4-9550-aa309ca4ed2a None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:00.668608 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-99999029-9a3a-4402-ac3a-aefd146fbf8f None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:00.670376 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-99999029-9a3a-4402-ac3a-aefd146fbf8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:00.783733 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 273/551] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:06:00.784643 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99999029-9a3a-4402-ac3a-aefd146fbf8f None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:00.787938 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:00.805266 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73149ae-bb87-40b4-9550-aa309ca4ed2a None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:00.882203 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45764c22-8f59-4027-88c4-1c44e715f3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:00.883802 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45764c22-8f59-4027-88c4-1c44e715f3c3 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:00.884325 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45764c22-8f59-4027-88c4-1c44e715f3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:00.884819 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45764c22-8f59-4027-88c4-1c44e715f3c3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:00.899585 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45764c22-8f59-4027-88c4-1c44e715f3c3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:00.907616 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45764c22-8f59-4027-88c4-1c44e715f3c3 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:00.928709 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:00.944624 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-45764c22-8f59-4027-88c4-1c44e715f3c3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 05 12:06:00.944624 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:00.944624 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:00.944624 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:00.944624 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:00.944624 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:00.944624 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:00.944624 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:00.944624 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:00.944624 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:00.944624 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:00.944624 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:00.944624 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:06:00.944624 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:06:00.944624 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:06:00.944624 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:06:00.945746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:00.945746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:00.945746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:06:00.945746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:00.945746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:06:00.945746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:00.945746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:00.945746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:00.945746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:00.945746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:00.945746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:00.945746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:00.945746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:00.945746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:00.945746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:00.945746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:00.945746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:00.945746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:00.946719 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:00.946719 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:00.946719 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:06:00.946719 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:06:00.946719 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:06:00.946719 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:06:00.946719 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:06:00.946719 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:06:00.946719 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 05 12:06:00.946719 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:00.958728 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 279/552] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:06:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:00.960085 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45764c22-8f59-4027-88c4-1c44e715f3c3 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:00.997193 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99999029-9a3a-4402-ac3a-aefd146fbf8f None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:01.088670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06dac80-eeaa-444e-a1c7-3713a2b28680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:01.089881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06dac80-eeaa-444e-a1c7-3713a2b28680 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:01.096079 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06dac80-eeaa-444e-a1c7-3713a2b28680 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:01.096943 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06dac80-eeaa-444e-a1c7-3713a2b28680 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:01.141425 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06dac80-eeaa-444e-a1c7-3713a2b28680 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:06:01.150234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-c06dac80-eeaa-444e-a1c7-3713a2b28680 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:06:01.150234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:01.150234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:01.150234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:01.150234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:01.150234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:01.150234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:01.150234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:01.150234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:01.150234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:01.150234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:01.150234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:01.150234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:06:01.150234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:06:01.150234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:06:01.150234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:06:01.151429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:06:01.151429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:06:01.151429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:06:01.151429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:06:01.151429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:01.151429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:01.151429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:01.151429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:01.151429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:01.151429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:01.151429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:01.151429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:01.151429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:01.151429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:01.151429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:01.151429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:01.152343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:01.152343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:01.152343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:01.152343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:01.152343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:06:01.152343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:06:01.152343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:06:01.152343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:06:01.152343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:06:01.152343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:06:01.152343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:06:01.152343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:01.154993 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 274/553] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Sep 5 12:06:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:01.156049 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06dac80-eeaa-444e-a1c7-3713a2b28680 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:01.282920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45764c22-8f59-4027-88c4-1c44e715f3c3 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:01.344127 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:01.349796 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06dac80-eeaa-444e-a1c7-3713a2b28680 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:01.356618 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af88541-9f60-4fab-9419-06e3b207c479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:01.358232 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af88541-9f60-4fab-9419-06e3b207c479 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:01.358682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af88541-9f60-4fab-9419-06e3b207c479 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:01.359122 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af88541-9f60-4fab-9419-06e3b207c479 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:01.360080 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af88541-9f60-4fab-9419-06e3b207c479 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:01.360572 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af88541-9f60-4fab-9419-06e3b207c479 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:01.362247 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af88541-9f60-4fab-9419-06e3b207c479 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:01.411332 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 280/554] 10.222.0.16 () {56 vars in 1135 bytes} [Mon Sep 5 12:06:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 452 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:01.412276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af88541-9f60-4fab-9419-06e3b207c479 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:01.441352 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d765d6-a241-48ac-8d7c-2b24d35e43fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:01.442379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d765d6-a241-48ac-8d7c-2b24d35e43fd None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:01.442807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d765d6-a241-48ac-8d7c-2b24d35e43fd None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:01.443343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d765d6-a241-48ac-8d7c-2b24d35e43fd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:01.464976 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:01.474673 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d765d6-a241-48ac-8d7c-2b24d35e43fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:06:01.485472 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-75d765d6-a241-48ac-8d7c-2b24d35e43fd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:06:01.485472 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:01.485472 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:01.485472 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:01.485472 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:01.485472 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:01.485472 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:01.485472 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:01.485472 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:01.485472 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:01.485472 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:01.485472 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:01.485472 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:06:01.485472 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:06:01.485472 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:06:01.485472 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:06:01.486684 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:06:01.486684 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:06:01.486684 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:06:01.486684 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:06:01.486684 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:01.486684 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:01.486684 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:01.486684 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:01.486684 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:01.486684 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:01.486684 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:01.486684 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:01.486684 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:01.486684 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:01.486684 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:01.486684 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:01.487800 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:01.487800 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:01.487800 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:01.487800 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:01.487800 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:06:01.487800 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:06:01.487800 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:06:01.487800 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:06:01.487800 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:06:01.487800 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:06:01.487800 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:06:01.487800 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:01.490589 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 275/555] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Sep 5 12:06:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:01.493021 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646db66-ed99-4ad6-91b3-1fe9eac0a1cb None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:01.493377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646db66-ed99-4ad6-91b3-1fe9eac0a1cb None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:01.493709 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646db66-ed99-4ad6-91b3-1fe9eac0a1cb None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:01.501235 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 276/556] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:01.501235 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646db66-ed99-4ad6-91b3-1fe9eac0a1cb None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:01.575116 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af88541-9f60-4fab-9419-06e3b207c479 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:01.629842 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0232b2a3-3c30-430a-9a8c-40d2b94f390a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:01.630503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0232b2a3-3c30-430a-9a8c-40d2b94f390a None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:01.630789 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0232b2a3-3c30-430a-9a8c-40d2b94f390a None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:01.631059 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0232b2a3-3c30-430a-9a8c-40d2b94f390a None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:01.631858 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0232b2a3-3c30-430a-9a8c-40d2b94f390a None admin] RBAC: Adding query filter params (name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:01.632207 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0232b2a3-3c30-430a-9a8c-40d2b94f390a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:01.633733 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0232b2a3-3c30-430a-9a8c-40d2b94f390a None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:01.644939 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 281/557] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Sep 5 12:06:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:01.645797 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0232b2a3-3c30-430a-9a8c-40d2b94f390a None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:01.659463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646db66-ed99-4ad6-91b3-1fe9eac0a1cb None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:01.703418 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d0093a-79e7-4605-b796-58f5c3817150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:01.704062 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0093a-79e7-4605-b796-58f5c3817150 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:01.704340 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0093a-79e7-4605-b796-58f5c3817150 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:01.704612 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0093a-79e7-4605-b796-58f5c3817150 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:01.721426 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:01.779025 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:01.781670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d0093a-79e7-4605-b796-58f5c3817150 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:01.783756 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d0093a-79e7-4605-b796-58f5c3817150 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:01.803587 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0232b2a3-3c30-430a-9a8c-40d2b94f390a None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:01.842618 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a41d80-c329-4bd4-a546-41f51db4cd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:01.843277 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a41d80-c329-4bd4-a546-41f51db4cd34 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:01.843497 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a41d80-c329-4bd4-a546-41f51db4cd34 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:01.843819 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a41d80-c329-4bd4-a546-41f51db4cd34 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:01.844660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a41d80-c329-4bd4-a546-41f51db4cd34 None admin] RBAC: Adding query filter params (name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:01.844924 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a41d80-c329-4bd4-a546-41f51db4cd34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:01.846472 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a41d80-c329-4bd4-a546-41f51db4cd34 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:01.855147 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 282/558] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Sep 5 12:06:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:01.856417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1566ec8-a332-4875-9a0b-664df4c411c2 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:01.856669 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1566ec8-a332-4875-9a0b-664df4c411c2 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:01.857425 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1566ec8-a332-4875-9a0b-664df4c411c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:01.877439 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-f1566ec8-a332-4875-9a0b-664df4c411c2 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:01.878782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f1566ec8-a332-4875-9a0b-664df4c411c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:01.916094 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 277/559] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:06:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:06:01.921199 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff05a4-1665-4043-9998-37f1cddb981d None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:01.921479 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff05a4-1665-4043-9998-37f1cddb981d None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:01.921752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff05a4-1665-4043-9998-37f1cddb981d None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:01.923066 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 278/560] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:01.923788 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff05a4-1665-4043-9998-37f1cddb981d None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:01.959779 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 283/561] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:06:01.960665 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1566ec8-a332-4875-9a0b-664df4c411c2 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:02.012789 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:02.047200 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:02.052816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff05a4-1665-4043-9998-37f1cddb981d None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:02.082418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1566ec8-a332-4875-9a0b-664df4c411c2 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:02.105301 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae22c2f-2063-4f55-aed4-daa2c87b10d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:02.107734 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae22c2f-2063-4f55-aed4-daa2c87b10d6 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:02.108172 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae22c2f-2063-4f55-aed4-daa2c87b10d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:02.108460 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae22c2f-2063-4f55-aed4-daa2c87b10d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:02.122966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae22c2f-2063-4f55-aed4-daa2c87b10d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:02.123552 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f7408-1d6e-402a-a929-72fcf6d3cd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:02.124414 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f7408-1d6e-402a-a929-72fcf6d3cd88 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:02.124536 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae22c2f-2063-4f55-aed4-daa2c87b10d6 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:02.125018 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f7408-1d6e-402a-a929-72fcf6d3cd88 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:02.125503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f7408-1d6e-402a-a929-72fcf6d3cd88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:02.143212 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-2ae22c2f-2063-4f55-aed4-daa2c87b10d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:02.143212 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:02.143212 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:02.143212 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:02.143212 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:02.143212 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:02.143212 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:02.143212 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:02.143212 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:02.143212 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:02.143212 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:02.143212 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:02.143212 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:02.143212 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:02.143212 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:02.143212 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:02.145096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:02.145096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:02.145096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:02.145096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:02.145096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:02.145096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:02.145096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:02.145096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:02.145096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:02.145096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:02.145096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:02.145096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:02.145096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:02.145096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:02.145096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:02.145096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:02.145096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:02.146522 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:02.146522 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:02.146522 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:02.146522 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 279/562] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:02.146522 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae22c2f-2063-4f55-aed4-daa2c87b10d6 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:02.149016 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4f7408-1d6e-402a-a929-72fcf6d3cd88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:02.152129 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4f7408-1d6e-402a-a929-72fcf6d3cd88 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:02.172695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-fa4f7408-1d6e-402a-a929-72fcf6d3cd88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:02.172695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:02.172695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:02.172695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:02.172695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:02.172695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:02.172695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:02.172695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:02.172695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:02.172695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:02.172695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:02.172695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:02.172695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:02.172695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:02.172695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:02.172695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:02.173758 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:02.173758 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:02.173758 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:02.173758 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:02.173758 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:02.173758 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:02.173758 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:02.173758 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:02.173758 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:02.173758 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:02.173758 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:02.173758 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:02.173758 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:02.173758 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:02.173758 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:02.173758 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:02.173758 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:02.183511 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:02.183511 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:02.183511 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:02.183511 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 284/563] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:02.183511 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611970e-55e0-4261-8a0c-e2ee5286a739 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:02.183511 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611970e-55e0-4261-8a0c-e2ee5286a739 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:02.183511 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611970e-55e0-4261-8a0c-e2ee5286a739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:02.202545 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-f611970e-55e0-4261-8a0c-e2ee5286a739 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:02.204481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f611970e-55e0-4261-8a0c-e2ee5286a739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:02.297943 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae22c2f-2063-4f55-aed4-daa2c87b10d6 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:02.304823 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 285/564] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:06:02.305733 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611970e-55e0-4261-8a0c-e2ee5286a739 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:02.343480 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:02.347267 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d220c9c3-326f-4fa4-b55e-078cd34febde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:02.348065 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d220c9c3-326f-4fa4-b55e-078cd34febde None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:02.348417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d220c9c3-326f-4fa4-b55e-078cd34febde None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:02.348701 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d220c9c3-326f-4fa4-b55e-078cd34febde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:02.366636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d220c9c3-326f-4fa4-b55e-078cd34febde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:02.368656 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d220c9c3-326f-4fa4-b55e-078cd34febde None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:02.389292 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-d220c9c3-326f-4fa4-b55e-078cd34febde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:02.389292 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:02.389292 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:02.389292 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:02.389292 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:02.389292 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:02.389292 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:02.389292 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:02.389292 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:02.389292 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:02.389292 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:02.389292 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:02.389292 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:02.389292 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:02.389292 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:02.389292 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:02.391147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:02.391147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:02.391147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:02.391147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:02.391147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:02.391147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:02.391147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:02.391147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:02.391147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:02.391147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:02.391147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:02.391147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:02.391147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:02.391147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:02.391147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:02.391147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:02.391147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:02.393704 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:02.393704 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:02.393704 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:02.396928 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 280/565] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:02.397964 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d220c9c3-326f-4fa4-b55e-078cd34febde None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:02.404376 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:02.446711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611970e-55e0-4261-8a0c-e2ee5286a739 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:02.502145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0ceeb-5d2c-40fd-85d1-76a807997757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:02.503595 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0ceeb-5d2c-40fd-85d1-76a807997757 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:02.504152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0ceeb-5d2c-40fd-85d1-76a807997757 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:02.504439 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0ceeb-5d2c-40fd-85d1-76a807997757 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:02.505133 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0ceeb-5d2c-40fd-85d1-76a807997757 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:02.505525 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0ceeb-5d2c-40fd-85d1-76a807997757 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:02.507704 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0ceeb-5d2c-40fd-85d1-76a807997757 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:02.525862 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 286/566] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:02.526828 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0ceeb-5d2c-40fd-85d1-76a807997757 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:02.535846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d220c9c3-326f-4fa4-b55e-078cd34febde None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:02.581108 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94225cd-c6ba-4da3-a585-fce5df96df05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:02.581935 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94225cd-c6ba-4da3-a585-fce5df96df05 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:02.582252 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94225cd-c6ba-4da3-a585-fce5df96df05 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:02.582701 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94225cd-c6ba-4da3-a585-fce5df96df05 None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:02.583831 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94225cd-c6ba-4da3-a585-fce5df96df05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:02.584344 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94225cd-c6ba-4da3-a585-fce5df96df05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:02.586098 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94225cd-c6ba-4da3-a585-fce5df96df05 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:02.607607 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 281/567] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:02.607607 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94225cd-c6ba-4da3-a585-fce5df96df05 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:02.645567 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:02.669829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0ceeb-5d2c-40fd-85d1-76a807997757 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:02.696032 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:02.705565 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c60c9-113f-4f86-8a48-3a34f11df2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:02.706120 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c60c9-113f-4f86-8a48-3a34f11df2ac None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:02.706394 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c60c9-113f-4f86-8a48-3a34f11df2ac None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:02.706665 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c60c9-113f-4f86-8a48-3a34f11df2ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:02.718760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c60c9-113f-4f86-8a48-3a34f11df2ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:06:02.724832 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-ce8c60c9-113f-4f86-8a48-3a34f11df2ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:06:02.724832 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:02.724832 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:02.724832 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:02.724832 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:02.724832 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:02.724832 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:02.724832 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:02.724832 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:02.724832 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:02.724832 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:02.724832 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:02.724832 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:06:02.724832 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:06:02.724832 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:06:02.724832 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:06:02.726194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:06:02.726194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:06:02.726194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:06:02.726194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:06:02.726194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:02.726194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:02.726194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:02.726194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:02.726194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:02.726194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:02.726194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:02.726194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:02.726194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:02.726194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:02.726194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:02.726194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:02.727358 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:02.727358 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:02.727358 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:02.727358 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:02.727358 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:06:02.727358 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:06:02.727358 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:06:02.727358 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:06:02.727358 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:06:02.727358 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:06:02.727358 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:06:02.727358 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:02.727358 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 287/568] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Sep 5 12:06:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:02.727358 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c60c9-113f-4f86-8a48-3a34f11df2ac None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:02.751692 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94225cd-c6ba-4da3-a585-fce5df96df05 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:02.790448 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715acb0f-0cb2-485e-b76e-c296b1d66913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:02.791046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715acb0f-0cb2-485e-b76e-c296b1d66913 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:02.791320 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715acb0f-0cb2-485e-b76e-c296b1d66913 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:02.791623 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715acb0f-0cb2-485e-b76e-c296b1d66913 None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:02.792276 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715acb0f-0cb2-485e-b76e-c296b1d66913 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:02.792607 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715acb0f-0cb2-485e-b76e-c296b1d66913 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:02.793939 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715acb0f-0cb2-485e-b76e-c296b1d66913 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:02.806789 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 282/569] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:02.807657 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715acb0f-0cb2-485e-b76e-c296b1d66913 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:02.861025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c60c9-113f-4f86-8a48-3a34f11df2ac None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:02.897351 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8f017-8f50-4c5a-ab59-f0c1823ac8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:02.897977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8f017-8f50-4c5a-ab59-f0c1823ac8ae None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:02.898247 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8f017-8f50-4c5a-ab59-f0c1823ac8ae None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:02.898510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8f017-8f50-4c5a-ab59-f0c1823ac8ae None admin] PATH_INFO: `/v3/users/glance` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:02.910322 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f8f017-8f50-4c5a-ab59-f0c1823ac8ae None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:02.913050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f8f017-8f50-4c5a-ab59-f0c1823ac8ae None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:02.922509 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715acb0f-0cb2-485e-b76e-c296b1d66913 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:02.928047 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-46f8f017-8f50-4c5a-ab59-f0c1823ac8ae None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 05 12:06:02.928047 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:02.928047 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:02.928047 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:02.928047 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:02.928047 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:02.928047 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:02.928047 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:02.928047 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:02.928047 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:02.928047 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:02.928047 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:02.928047 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:06:02.928047 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:06:02.928047 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:06:02.928047 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:06:02.929030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:02.929030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:02.929030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:06:02.929030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:02.929030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:06:02.929030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:02.929030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:02.929030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:02.929030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:02.929030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:02.929030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:02.929030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:02.929030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:02.929030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:02.929030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:02.929030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:02.929030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:02.929030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:02.929982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:02.929982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:02.929982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:06:02.929982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:06:02.929982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:06:02.929982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:06:02.929982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:06:02.929982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:06:02.929982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 05 12:06:02.929982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:02.929982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 288/570] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:06:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:02.930591 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8f017-8f50-4c5a-ab59-f0c1823ac8ae None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:02.946194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:02.951272 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:02.964256 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa08a84-0c9d-45f0-902e-80e6f90088df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:02.965264 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa08a84-0c9d-45f0-902e-80e6f90088df None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:02.965588 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa08a84-0c9d-45f0-902e-80e6f90088df None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:02.965865 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa08a84-0c9d-45f0-902e-80e6f90088df None admin] PATH_INFO: `/v3/roles/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:02.980702 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa08a84-0c9d-45f0-902e-80e6f90088df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:06:02.989382 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-dfa08a84-0c9d-45f0-902e-80e6f90088df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:06:02.989382 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:02.989382 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:02.989382 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:02.989382 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:02.989382 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:02.989382 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:02.989382 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:02.989382 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:02.989382 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:02.989382 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:02.989382 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:02.989382 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:06:02.989382 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:06:02.989382 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:06:02.989382 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:06:02.990672 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:06:02.990672 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:06:02.990672 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:06:02.990672 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:06:02.990672 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:02.990672 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:02.990672 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:02.990672 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:02.990672 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:02.990672 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:02.990672 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:02.990672 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:02.990672 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:02.990672 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:02.990672 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:02.990672 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:02.991757 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:02.991757 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:02.991757 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:02.991757 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:02.991757 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:06:02.991757 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:06:02.991757 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:06:02.991757 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:06:02.991757 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:06:02.991757 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:06:02.991757 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:06:02.991757 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:02.991757 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 283/571] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Sep 5 12:06:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:02.992533 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa08a84-0c9d-45f0-902e-80e6f90088df None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:03.068729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8f017-8f50-4c5a-ab59-f0c1823ac8ae None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:03.108981 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa08a84-0c9d-45f0-902e-80e6f90088df None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:03.110612 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6896791-a8bc-4e7f-abb1-fafc34dfbe15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:03.111669 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6896791-a8bc-4e7f-abb1-fafc34dfbe15 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:03.112035 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6896791-a8bc-4e7f-abb1-fafc34dfbe15 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:03.112508 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6896791-a8bc-4e7f-abb1-fafc34dfbe15 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:03.125206 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6896791-a8bc-4e7f-abb1-fafc34dfbe15 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:03.127798 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6896791-a8bc-4e7f-abb1-fafc34dfbe15 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:03.144271 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-d6896791-a8bc-4e7f-abb1-fafc34dfbe15 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:06:03.144271 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:03.144271 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:03.144271 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:03.144271 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:03.144271 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:03.144271 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:03.144271 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:03.144271 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:03.144271 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:03.144271 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:03.144271 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:03.144271 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:06:03.144271 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:06:03.144271 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:06:03.144271 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:06:03.145599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:03.145599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:03.145599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:06:03.145599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:03.145599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:06:03.145599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:03.145599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:03.145599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:03.145599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:03.145599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:03.145599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:03.145599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:03.145599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:03.145599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:03.145599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:03.145599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:03.145599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:03.145599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:03.146704 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:03.146704 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:03.146704 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:06:03.146704 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:06:03.146704 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:06:03.146704 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:06:03.146704 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:06:03.146704 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:06:03.146704 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:06:03.146704 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:03.146704 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 289/572] 10.222.0.16 () {56 vars in 1093 bytes} [Mon Sep 5 12:06:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:03.147435 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6896791-a8bc-4e7f-abb1-fafc34dfbe15 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:03.158660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab220e-a910-4fa7-9307-d464bd138105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:03.159495 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab220e-a910-4fa7-9307-d464bd138105 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:03.159738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab220e-a910-4fa7-9307-d464bd138105 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:03.160041 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab220e-a910-4fa7-9307-d464bd138105 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:03.160962 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab220e-a910-4fa7-9307-d464bd138105 None admin] RBAC: Adding query filter params (name=service) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:03.161261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab220e-a910-4fa7-9307-d464bd138105 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:03.162664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab220e-a910-4fa7-9307-d464bd138105 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:03.172753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 284/573] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Sep 5 12:06:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:03.173688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab220e-a910-4fa7-9307-d464bd138105 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:03.226720 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:03.242202 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:03.273315 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6896791-a8bc-4e7f-abb1-fafc34dfbe15 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:03.310739 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395f259-a400-4a82-aee0-7efb44718c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:03.311371 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395f259-a400-4a82-aee0-7efb44718c6d None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:03.311684 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395f259-a400-4a82-aee0-7efb44718c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:03.311955 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395f259-a400-4a82-aee0-7efb44718c6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:03.318747 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4395f259-a400-4a82-aee0-7efb44718c6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:03.320142 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab220e-a910-4fa7-9307-d464bd138105 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:03.320268 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4395f259-a400-4a82-aee0-7efb44718c6d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:03.331748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-4395f259-a400-4a82-aee0-7efb44718c6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:03.331748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:03.331748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:03.331748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:03.331748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:03.331748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:03.331748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:03.331748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:03.331748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:03.331748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:03.331748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:03.331748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:03.331748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:06:03.331748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:06:03.331748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:06:03.332927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:06:03.332927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:03.332927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:03.332927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:03.332927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:03.332927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:03.332927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:03.332927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:03.332927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:03.332927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:03.332927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:03.332927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:03.332927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:03.332927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:03.332927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:03.332927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:03.332927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:03.333835 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:06:03.333835 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:06:03.333835 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:06:03.333835 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:06:03.333835 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:06:03.333835 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:06:03.333835 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:03.333835 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:03.333835 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 290/574] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:06:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:03.334348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395f259-a400-4a82-aee0-7efb44718c6d None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:03.377835 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f3fbd-6797-4158-a9ae-8b558d2adbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:03.378442 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f3fbd-6797-4158-a9ae-8b558d2adbea None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:03.378725 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f3fbd-6797-4158-a9ae-8b558d2adbea None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:03.378998 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f3fbd-6797-4158-a9ae-8b558d2adbea None admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:03.379776 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f3fbd-6797-4158-a9ae-8b558d2adbea None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:03.380080 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f3fbd-6797-4158-a9ae-8b558d2adbea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:03.381993 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f3fbd-6797-4158-a9ae-8b558d2adbea None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:03.401390 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 285/575] 10.222.0.16 () {56 vars in 1137 bytes} [Mon Sep 5 12:06:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:03.402247 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f3fbd-6797-4158-a9ae-8b558d2adbea None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:03.461898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395f259-a400-4a82-aee0-7efb44718c6d None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:03.490783 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:03.502245 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e1f4f-1ecf-4163-8d1e-8f0991748f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:03.502946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e1f4f-1ecf-4163-8d1e-8f0991748f8f None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:03.503235 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e1f4f-1ecf-4163-8d1e-8f0991748f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:03.503500 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e1f4f-1ecf-4163-8d1e-8f0991748f8f None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:03.504203 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e1f4f-1ecf-4163-8d1e-8f0991748f8f None admin] RBAC: Adding query filter params (name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:03.504505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e1f4f-1ecf-4163-8d1e-8f0991748f8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:03.505934 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e1f4f-1ecf-4163-8d1e-8f0991748f8f None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:03.515627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 291/576] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Sep 5 12:06:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:03.516790 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e1f4f-1ecf-4163-8d1e-8f0991748f8f None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:03.527089 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f3fbd-6797-4158-a9ae-8b558d2adbea None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:03.531950 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:03.570074 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a1538-d371-4b29-aff3-498050d8ba03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:03.570686 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a1538-d371-4b29-aff3-498050d8ba03 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:03.570966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a1538-d371-4b29-aff3-498050d8ba03 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:03.571236 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a1538-d371-4b29-aff3-498050d8ba03 None admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:03.572024 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a1538-d371-4b29-aff3-498050d8ba03 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:03.572330 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a1538-d371-4b29-aff3-498050d8ba03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:03.573760 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a1538-d371-4b29-aff3-498050d8ba03 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:03.595349 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 286/577] 10.222.0.16 () {56 vars in 1133 bytes} [Mon Sep 5 12:06:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:03.596354 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a1538-d371-4b29-aff3-498050d8ba03 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:03.634126 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e1f4f-1ecf-4163-8d1e-8f0991748f8f None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:03.668158 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6825682-2390-4b15-940b-8d20bf17da53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:03.668787 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6825682-2390-4b15-940b-8d20bf17da53 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:03.669070 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6825682-2390-4b15-940b-8d20bf17da53 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:03.669400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6825682-2390-4b15-940b-8d20bf17da53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:03.682564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6825682-2390-4b15-940b-8d20bf17da53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:03.684603 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6825682-2390-4b15-940b-8d20bf17da53 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:03.702833 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-c6825682-2390-4b15-940b-8d20bf17da53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:03.702833 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:03.702833 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:03.702833 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:03.702833 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:03.702833 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:03.702833 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:03.702833 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:03.702833 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:03.702833 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:03.702833 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:03.702833 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:03.702833 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:03.702833 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:03.702833 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:03.702833 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:03.704042 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:03.704042 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:03.704042 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:03.704042 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:03.704042 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:03.704042 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:03.704042 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:03.704042 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:03.704042 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:03.704042 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:03.704042 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:03.704042 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:03.704042 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:03.704042 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:03.704042 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:03.704042 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:03.704042 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:03.705118 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:03.705118 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:03.705118 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:03.705118 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 292/578] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:03.705118 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6825682-2390-4b15-940b-8d20bf17da53 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:03.705561 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a1538-d371-4b29-aff3-498050d8ba03 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:03.743329 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0bafbf-df78-493e-9fd6-63ec4acdb0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:03.743880 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bafbf-df78-493e-9fd6-63ec4acdb0a4 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:03.744153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bafbf-df78-493e-9fd6-63ec4acdb0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:03.744489 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bafbf-df78-493e-9fd6-63ec4acdb0a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:03.745128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0bafbf-df78-493e-9fd6-63ec4acdb0a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:03.745434 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0bafbf-df78-493e-9fd6-63ec4acdb0a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:03.746864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0bafbf-df78-493e-9fd6-63ec4acdb0a4 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:03.759150 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 287/579] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:06:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:03.760039 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0bafbf-df78-493e-9fd6-63ec4acdb0a4 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:03.775208 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:03.797813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:03.854726 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6825682-2390-4b15-940b-8d20bf17da53 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:03.881495 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0bafbf-df78-493e-9fd6-63ec4acdb0a4 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:03.896818 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e11d8d-4407-4cc4-8161-9d90bbaf9fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:03.897661 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e11d8d-4407-4cc4-8161-9d90bbaf9fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:03.898001 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e11d8d-4407-4cc4-8161-9d90bbaf9fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:03.898318 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e11d8d-4407-4cc4-8161-9d90bbaf9fe4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:03.913039 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e11d8d-4407-4cc4-8161-9d90bbaf9fe4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:03.915066 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e11d8d-4407-4cc4-8161-9d90bbaf9fe4 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:03.931688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce226e-07c5-4285-a11f-ac3b5db608bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:03.932335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce226e-07c5-4285-a11f-ac3b5db608bf None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:03.932646 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce226e-07c5-4285-a11f-ac3b5db608bf None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:03.933073 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce226e-07c5-4285-a11f-ac3b5db608bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:03.934320 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-06e11d8d-4407-4cc4-8161-9d90bbaf9fe4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:03.934320 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:03.934320 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:03.934320 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:03.934320 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:03.934320 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:03.934320 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:03.934320 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:03.934320 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:03.934320 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:03.934320 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:03.934320 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:03.934320 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:03.934320 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:03.934320 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:03.934320 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:03.935356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:03.935356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:03.935356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:03.935356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:03.935356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:03.935356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:03.935356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:03.935356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:03.935356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:03.935356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:03.935356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:03.935356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:03.935356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:03.935356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:03.935356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:03.935356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:03.935356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:03.936402 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:03.936402 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:03.936402 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:03.936402 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 293/580] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:03.936402 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e11d8d-4407-4cc4-8161-9d90bbaf9fe4 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:03.949174 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cce226e-07c5-4285-a11f-ac3b5db608bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:03.951338 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cce226e-07c5-4285-a11f-ac3b5db608bf None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:03.969121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-1cce226e-07c5-4285-a11f-ac3b5db608bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:03.969121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:03.969121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:03.969121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:03.969121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:03.969121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:03.969121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:03.969121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:03.969121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:03.969121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:03.969121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:03.969121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:03.969121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:03.969121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:03.969121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:03.969121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:03.970405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:03.970405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:03.970405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:03.970405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:03.970405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:03.970405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:03.970405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:03.970405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:03.970405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:03.970405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:03.970405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:03.970405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:03.970405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:03.970405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:03.970405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:03.970405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:03.970405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:03.971393 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:03.971393 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:03.971393 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:03.971393 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 288/581] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:03.971393 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce226e-07c5-4285-a11f-ac3b5db608bf None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:04.060517 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:04.066850 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:04.072037 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e11d8d-4407-4cc4-8161-9d90bbaf9fe4 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:04.097375 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce226e-07c5-4285-a11f-ac3b5db608bf None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:04.111830 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c11b54d-ccda-4347-ac12-bc4f4f4d6550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:04.112498 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11b54d-ccda-4347-ac12-bc4f4f4d6550 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:04.112783 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11b54d-ccda-4347-ac12-bc4f4f4d6550 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:04.113060 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11b54d-ccda-4347-ac12-bc4f4f4d6550 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:04.124947 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c11b54d-ccda-4347-ac12-bc4f4f4d6550 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:04.126439 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c11b54d-ccda-4347-ac12-bc4f4f4d6550 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:04.142488 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753b19a-7cc2-41be-b777-37e299cd707e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:04.143444 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753b19a-7cc2-41be-b777-37e299cd707e None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:04.143949 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753b19a-7cc2-41be-b777-37e299cd707e None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:04.144410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753b19a-7cc2-41be-b777-37e299cd707e None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:04.145510 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1753b19a-7cc2-41be-b777-37e299cd707e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:04.146021 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1753b19a-7cc2-41be-b777-37e299cd707e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:04.148450 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1753b19a-7cc2-41be-b777-37e299cd707e None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:04.149760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-4c11b54d-ccda-4347-ac12-bc4f4f4d6550 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:04.149760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:04.149760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:04.149760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:04.149760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:04.149760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:04.149760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:04.149760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:04.149760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:04.149760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:04.149760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:04.149760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:04.149760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:04.149760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:04.149760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:04.149760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:04.150839 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:04.150839 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:04.150839 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:04.150839 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:04.150839 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:04.150839 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:04.150839 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:04.150839 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:04.150839 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:04.150839 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:04.150839 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:04.150839 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:04.150839 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:04.150839 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:04.150839 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:04.150839 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:04.150839 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:04.151906 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:04.151906 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:04.151906 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:04.151906 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 294/582] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:04.153305 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c11b54d-ccda-4347-ac12-bc4f4f4d6550 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:04.166525 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 289/583] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:04.167723 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753b19a-7cc2-41be-b777-37e299cd707e None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:04.267779 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c11b54d-ccda-4347-ac12-bc4f4f4d6550 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:04.280410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753b19a-7cc2-41be-b777-37e299cd707e None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:04.304202 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475e06e-1caf-4a1b-b7bb-6059ad034424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:04.304821 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475e06e-1caf-4a1b-b7bb-6059ad034424 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:04.305100 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475e06e-1caf-4a1b-b7bb-6059ad034424 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:04.305371 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475e06e-1caf-4a1b-b7bb-6059ad034424 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:04.306115 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475e06e-1caf-4a1b-b7bb-6059ad034424 None admin] RBAC: Adding query filter params (role.id=8cf764a0144a490a822fcf86d7aceeb0, scope.project.id=fa3178684d184ec880f17640d19d55b7, user.id=857eb7657e7d45938baca17f91a16901) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:04.306425 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475e06e-1caf-4a1b-b7bb-6059ad034424 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:04.307936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475e06e-1caf-4a1b-b7bb-6059ad034424 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:04.314616 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 295/584] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:06:04 2022] GET /identity/v3/role_assignments?user.id=857eb7657e7d45938baca17f91a16901&scope.project.id=fa3178684d184ec880f17640d19d55b7&role.id=8cf764a0144a490a822fcf86d7aceeb0 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:04.315382 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475e06e-1caf-4a1b-b7bb-6059ad034424 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:04.317830 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ec113-65bf-4145-9d1f-9b1218d43507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:04.318453 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ec113-65bf-4145-9d1f-9b1218d43507 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:04.318730 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ec113-65bf-4145-9d1f-9b1218d43507 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:04.319001 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ec113-65bf-4145-9d1f-9b1218d43507 None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:04.319716 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ec113-65bf-4145-9d1f-9b1218d43507 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:04.320018 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ec113-65bf-4145-9d1f-9b1218d43507 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:04.321485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ec113-65bf-4145-9d1f-9b1218d43507 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:04.323300 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:04.331324 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:04.344799 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 290/585] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:04.344799 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ec113-65bf-4145-9d1f-9b1218d43507 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:04.452650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ec113-65bf-4145-9d1f-9b1218d43507 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:04.452911 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475e06e-1caf-4a1b-b7bb-6059ad034424 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:04.490916 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97281a2a-f715-470d-b139-fdd9abca8afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:04.491543 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97281a2a-f715-470d-b139-fdd9abca8afa None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:04.491826 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97281a2a-f715-470d-b139-fdd9abca8afa None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:04.492097 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97281a2a-f715-470d-b139-fdd9abca8afa None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:04.492746 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97281a2a-f715-470d-b139-fdd9abca8afa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:04.493050 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97281a2a-f715-470d-b139-fdd9abca8afa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:04.493855 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5737cc-52ca-4ae8-9471-fc5e2e8c4b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:04.494414 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97281a2a-f715-470d-b139-fdd9abca8afa None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:04.494568 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5737cc-52ca-4ae8-9471-fc5e2e8c4b94 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:04.494906 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5737cc-52ca-4ae8-9471-fc5e2e8c4b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:04.495248 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5737cc-52ca-4ae8-9471-fc5e2e8c4b94 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:04.496010 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5737cc-52ca-4ae8-9471-fc5e2e8c4b94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:04.496371 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5737cc-52ca-4ae8-9471-fc5e2e8c4b94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:04.498350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5737cc-52ca-4ae8-9471-fc5e2e8c4b94 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:04.507891 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 291/586] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:04.511217 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 296/587] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:04.513862 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97281a2a-f715-470d-b139-fdd9abca8afa None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:04.516867 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5737cc-52ca-4ae8-9471-fc5e2e8c4b94 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:04.571590 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:04.594406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:04.628255 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97281a2a-f715-470d-b139-fdd9abca8afa None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:04.635645 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5737cc-52ca-4ae8-9471-fc5e2e8c4b94 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:04.668880 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66f468-7067-4c15-8d25-5aebf0c52108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:04.670070 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66f468-7067-4c15-8d25-5aebf0c52108 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:04.670351 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66f468-7067-4c15-8d25-5aebf0c52108 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:04.670626 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66f468-7067-4c15-8d25-5aebf0c52108 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:04.677374 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259ef2e-a231-4b54-87f7-62dc86e505a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:04.678328 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259ef2e-a231-4b54-87f7-62dc86e505a1 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:04.678608 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259ef2e-a231-4b54-87f7-62dc86e505a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:04.678879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259ef2e-a231-4b54-87f7-62dc86e505a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:04.679261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be66f468-7067-4c15-8d25-5aebf0c52108 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:04.680802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be66f468-7067-4c15-8d25-5aebf0c52108 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:04.687468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259ef2e-a231-4b54-87f7-62dc86e505a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:04.688977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259ef2e-a231-4b54-87f7-62dc86e505a1 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:04.694520 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-be66f468-7067-4c15-8d25-5aebf0c52108 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 05 12:06:04.694520 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:04.694520 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:04.694520 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:04.694520 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:04.694520 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:04.694520 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:04.694520 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:04.694520 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:04.694520 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:04.694520 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:04.694520 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:04.694520 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:06:04.694520 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:06:04.694520 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:06:04.694520 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:06:04.695586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:04.695586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:04.695586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:06:04.695586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:04.695586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:06:04.695586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:04.695586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:04.695586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:04.695586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:04.695586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:04.695586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:04.695586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:04.695586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:04.695586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:04.695586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:04.695586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:04.695586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:04.695586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:04.696555 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:04.696555 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:04.696555 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:06:04.696555 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:06:04.696555 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:06:04.696555 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:06:04.696555 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:06:04.696555 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:06:04.696555 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 05 12:06:04.696555 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:04.696555 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 292/588] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Sep 5 12:06:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:04.697158 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66f468-7067-4c15-8d25-5aebf0c52108 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:04.704729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-4259ef2e-a231-4b54-87f7-62dc86e505a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:04.704729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:04.704729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:04.704729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:04.704729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:04.704729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:04.704729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:04.704729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:04.704729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:04.704729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:04.704729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:04.704729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:04.704729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:06:04.704729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:06:04.704729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:06:04.705726 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:06:04.705726 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:04.705726 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:04.705726 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:04.705726 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:04.705726 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:04.705726 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:04.705726 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:04.705726 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:04.705726 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:04.705726 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:04.705726 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:04.705726 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:04.705726 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:04.705726 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:04.705726 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:04.705726 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:04.712147 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:06:04.712147 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:06:04.712147 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:06:04.712147 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:06:04.712147 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:06:04.712147 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:06:04.712147 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:04.712147 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:04.712147 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 297/589] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:06:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:04.712147 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259ef2e-a231-4b54-87f7-62dc86e505a1 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:04.838362 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66f468-7067-4c15-8d25-5aebf0c52108 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:04.846566 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259ef2e-a231-4b54-87f7-62dc86e505a1 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:04.855215 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:04.879737 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe016310-9b60-4b3a-85ee-944df4f3a07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:04.880522 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe016310-9b60-4b3a-85ee-944df4f3a07c None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:04.880683 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:04.880986 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe016310-9b60-4b3a-85ee-944df4f3a07c None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:04.881356 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe016310-9b60-4b3a-85ee-944df4f3a07c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:04.898439 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe016310-9b60-4b3a-85ee-944df4f3a07c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:04.900080 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe016310-9b60-4b3a-85ee-944df4f3a07c None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:04.900205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301fbaf6-2b54-4ec2-b4ae-6dff421d93fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:04.900829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fbaf6-2b54-4ec2-b4ae-6dff421d93fc None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:04.901110 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fbaf6-2b54-4ec2-b4ae-6dff421d93fc None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:04.901380 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fbaf6-2b54-4ec2-b4ae-6dff421d93fc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:04.909714 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301fbaf6-2b54-4ec2-b4ae-6dff421d93fc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:04.911281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301fbaf6-2b54-4ec2-b4ae-6dff421d93fc None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:04.913396 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-fe016310-9b60-4b3a-85ee-944df4f3a07c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:04.913396 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:04.913396 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:04.913396 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:04.913396 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:04.913396 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:04.913396 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:04.913396 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:04.913396 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:04.913396 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:04.913396 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:04.913396 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:04.913396 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:06:04.913396 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:06:04.913396 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:06:04.914506 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:06:04.914506 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:04.914506 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:04.914506 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:04.914506 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:04.914506 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:04.914506 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:04.914506 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:04.914506 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:04.914506 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:04.914506 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:04.914506 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:04.914506 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:04.914506 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:04.914506 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:04.914506 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:04.914506 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:04.915628 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:06:04.915628 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:06:04.915628 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:06:04.915628 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:06:04.915628 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:06:04.915628 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:06:04.915628 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:04.915628 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:04.919249 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 293/590] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:06:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:04.920557 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe016310-9b60-4b3a-85ee-944df4f3a07c None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:04.927168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-301fbaf6-2b54-4ec2-b4ae-6dff421d93fc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 12:06:04.927168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:04.927168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:04.927168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:04.927168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:04.927168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:04.927168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:04.927168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:04.927168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:04.927168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:04.927168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:04.927168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:04.927168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:06:04.927168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:06:04.927168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:06:04.927168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:06:04.928453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:04.928453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:04.928453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:06:04.928453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:04.928453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:06:04.928453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:04.928453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:04.928453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:04.928453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:04.928453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:04.928453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:04.928453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:04.928453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:04.928453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:04.928453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:04.928453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:04.928453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:04.928453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:04.929761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:04.929761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:04.929761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:06:04.929761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:06:04.929761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:06:04.929761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:06:04.929761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:06:04.929761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:06:04.929761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 12:06:04.929761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:04.935902 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 298/591] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:06:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:04.936710 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301fbaf6-2b54-4ec2-b4ae-6dff421d93fc None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:05.032272 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe016310-9b60-4b3a-85ee-944df4f3a07c None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:05.046096 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301fbaf6-2b54-4ec2-b4ae-6dff421d93fc None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:05.069835 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f05ddd-a37b-41fe-9fdc-9410bc4b44be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:05.070591 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f05ddd-a37b-41fe-9fdc-9410bc4b44be None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:05.070964 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f05ddd-a37b-41fe-9fdc-9410bc4b44be None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:05.071329 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f05ddd-a37b-41fe-9fdc-9410bc4b44be None admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:05.072160 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f05ddd-a37b-41fe-9fdc-9410bc4b44be None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:05.072526 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f05ddd-a37b-41fe-9fdc-9410bc4b44be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:05.074027 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f05ddd-a37b-41fe-9fdc-9410bc4b44be None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:05.081420 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828863a-4266-4aca-9ae1-02115671f70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:05.082113 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828863a-4266-4aca-9ae1-02115671f70a None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:05.082532 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828863a-4266-4aca-9ae1-02115671f70a None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:05.082793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828863a-4266-4aca-9ae1-02115671f70a None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:05.083567 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828863a-4266-4aca-9ae1-02115671f70a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:05.083944 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828863a-4266-4aca-9ae1-02115671f70a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:05.085465 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828863a-4266-4aca-9ae1-02115671f70a None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:05.098511 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 299/592] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:06:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:05.099079 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 294/593] 10.222.0.16 () {56 vars in 1139 bytes} [Mon Sep 5 12:06:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:05.099602 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828863a-4266-4aca-9ae1-02115671f70a None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:05.100513 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f05ddd-a37b-41fe-9fdc-9410bc4b44be None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:05.138094 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:05.157772 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:05.212060 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828863a-4266-4aca-9ae1-02115671f70a None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:05.213134 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f05ddd-a37b-41fe-9fdc-9410bc4b44be None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:05.250395 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f127b6-d5f6-4256-803a-70f865e38f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:05.251082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f127b6-d5f6-4256-803a-70f865e38f29 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:05.251356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f127b6-d5f6-4256-803a-70f865e38f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:05.251658 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f127b6-d5f6-4256-803a-70f865e38f29 None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:05.252503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f127b6-d5f6-4256-803a-70f865e38f29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:05.252616 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2eb55-04f6-48d4-8746-b6fd2eed6f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:05.252810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f127b6-d5f6-4256-803a-70f865e38f29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:05.253112 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2eb55-04f6-48d4-8746-b6fd2eed6f79 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:05.253387 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2eb55-04f6-48d4-8746-b6fd2eed6f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:05.253653 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2eb55-04f6-48d4-8746-b6fd2eed6f79 None admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:05.254245 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f127b6-d5f6-4256-803a-70f865e38f29 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:05.254418 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e2eb55-04f6-48d4-8746-b6fd2eed6f79 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:05.254721 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e2eb55-04f6-48d4-8746-b6fd2eed6f79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:05.256412 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e2eb55-04f6-48d4-8746-b6fd2eed6f79 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:05.267095 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 300/594] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:06:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:05.268041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f127b6-d5f6-4256-803a-70f865e38f29 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:05.281901 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 295/595] 10.222.0.16 () {56 vars in 1137 bytes} [Mon Sep 5 12:06:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:05.282607 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2eb55-04f6-48d4-8746-b6fd2eed6f79 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:05.385120 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:05.385316 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f127b6-d5f6-4256-803a-70f865e38f29 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:05.395723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:05.397939 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2eb55-04f6-48d4-8746-b6fd2eed6f79 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:05.432101 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e188f-e121-41a5-9ac8-5e5a2115053c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:05.432836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e188f-e121-41a5-9ac8-5e5a2115053c None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:05.433004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e188f-e121-41a5-9ac8-5e5a2115053c None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:05.433274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e188f-e121-41a5-9ac8-5e5a2115053c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:05.439292 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a892c67-6ba9-4810-963a-a68383cdcb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:05.440700 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a892c67-6ba9-4810-963a-a68383cdcb94 None admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:05.441189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a892c67-6ba9-4810-963a-a68383cdcb94 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:05.441639 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a892c67-6ba9-4810-963a-a68383cdcb94 None admin] PATH_INFO: `/v3/projects/fa3178684d184ec880f17640d19d55b7/users/ed75c0be809d4a0eb0e2ccc59805b1ad/roles/8cf764a0144a490a822fcf86d7aceeb0` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:05.444442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5e188f-e121-41a5-9ac8-5e5a2115053c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:05.445912 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5e188f-e121-41a5-9ac8-5e5a2115053c None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:05.459607 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a892c67-6ba9-4810-963a-a68383cdcb94 None admin] RBAC: Authorizing `identity:create_grant(project_id=fa3178684d184ec880f17640d19d55b7, user_id=ed75c0be809d4a0eb0e2ccc59805b1ad, role_id=8cf764a0144a490a822fcf86d7aceeb0)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:05.460886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-8c5e188f-e121-41a5-9ac8-5e5a2115053c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:05.460886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:05.460886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:05.460886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:05.460886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:05.460886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:05.460886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:05.460886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:05.460886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:05.460886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:05.460886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:05.460886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:05.460886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:05.460886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:05.460886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:05.460886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:05.461829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:05.461829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:05.461829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:05.461829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:05.461829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:05.461829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:05.461829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:05.461829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:05.461829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:05.461829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:05.461829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:05.461829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:05.461829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:05.461829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:05.461829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:05.461829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:05.461829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:05.462800 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:05.462800 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:05.462800 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:05.462973 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a892c67-6ba9-4810-963a-a68383cdcb94 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:05.463044 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 301/596] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:05.463044 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e188f-e121-41a5-9ac8-5e5a2115053c None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:05.480101 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 296/597] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Sep 5 12:06:05 2022] PUT /identity/v3/projects/fa3178684d184ec880f17640d19d55b7/users/ed75c0be809d4a0eb0e2ccc59805b1ad/roles/8cf764a0144a490a822fcf86d7aceeb0 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:06:05.481923 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a892c67-6ba9-4810-963a-a68383cdcb94 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:05.592645 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e188f-e121-41a5-9ac8-5e5a2115053c None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:05.638604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a892c67-6ba9-4810-963a-a68383cdcb94 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:05.641573 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02563dcd-b592-4856-8438-6923b077d0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:05.642556 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02563dcd-b592-4856-8438-6923b077d0ec None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:05.642935 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02563dcd-b592-4856-8438-6923b077d0ec None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:05.643306 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02563dcd-b592-4856-8438-6923b077d0ec None admin] PATH_INFO: `/v3/projects/fa3178684d184ec880f17640d19d55b7/users/7cda290418504f0db85d665479a05353/roles/8cf764a0144a490a822fcf86d7aceeb0` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:05.660245 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02563dcd-b592-4856-8438-6923b077d0ec None admin] RBAC: Authorizing `identity:create_grant(project_id=fa3178684d184ec880f17640d19d55b7, user_id=7cda290418504f0db85d665479a05353, role_id=8cf764a0144a490a822fcf86d7aceeb0)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:05.661701 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02563dcd-b592-4856-8438-6923b077d0ec None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:05.684980 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 302/598] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Sep 5 12:06:05 2022] PUT /identity/v3/projects/fa3178684d184ec880f17640d19d55b7/users/7cda290418504f0db85d665479a05353/roles/8cf764a0144a490a822fcf86d7aceeb0 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:06:05.685349 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02563dcd-b592-4856-8438-6923b077d0ec None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:05.691797 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdad6d-e1a3-4623-8cb8-7accc48596f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:05.692511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdad6d-e1a3-4623-8cb8-7accc48596f9 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:05.692787 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdad6d-e1a3-4623-8cb8-7accc48596f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:05.693111 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdad6d-e1a3-4623-8cb8-7accc48596f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:05.701342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:05.709117 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:05.712807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bdad6d-e1a3-4623-8cb8-7accc48596f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:05.715509 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bdad6d-e1a3-4623-8cb8-7accc48596f9 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:05.739515 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-e0bdad6d-e1a3-4623-8cb8-7accc48596f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:05.739515 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:05.739515 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:05.739515 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:05.739515 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:05.739515 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:05.739515 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:05.739515 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:05.739515 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:05.739515 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:05.739515 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:05.739515 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:05.739515 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:05.739515 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:05.739515 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:05.739515 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:05.740552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:05.740552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:05.740552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:05.740552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:05.740552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:05.740552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:05.740552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:05.740552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:05.740552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:05.740552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:05.740552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:05.740552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:05.740552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:05.740552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:05.740552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:05.740552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:05.740552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:05.741593 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:05.741593 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:05.741593 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:05.743700 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 297/599] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:05.749100 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdad6d-e1a3-4623-8cb8-7accc48596f9 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:05.889430 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02563dcd-b592-4856-8438-6923b077d0ec None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:05.933100 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ddb6a-0ae7-4cb5-9d66-ca4ee440aea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:05.933726 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ddb6a-0ae7-4cb5-9d66-ca4ee440aea5 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:05.934005 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ddb6a-0ae7-4cb5-9d66-ca4ee440aea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:05.934275 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ddb6a-0ae7-4cb5-9d66-ca4ee440aea5 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:05.935459 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ddb6a-0ae7-4cb5-9d66-ca4ee440aea5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:05.935943 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ddb6a-0ae7-4cb5-9d66-ca4ee440aea5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:05.937409 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ddb6a-0ae7-4cb5-9d66-ca4ee440aea5 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:05.945070 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdad6d-e1a3-4623-8cb8-7accc48596f9 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:05.950416 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 303/600] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:05.956414 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ddb6a-0ae7-4cb5-9d66-ca4ee440aea5 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:05.983759 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3336056-007b-4b36-8862-ee7900be6c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:05.984370 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3336056-007b-4b36-8862-ee7900be6c98 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:05.984645 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3336056-007b-4b36-8862-ee7900be6c98 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:05.984998 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3336056-007b-4b36-8862-ee7900be6c98 None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:05.985659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3336056-007b-4b36-8862-ee7900be6c98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:05.985989 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3336056-007b-4b36-8862-ee7900be6c98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:05.987355 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3336056-007b-4b36-8862-ee7900be6c98 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:06.006909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 298/601] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:06.008401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3336056-007b-4b36-8862-ee7900be6c98 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:06.034057 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:06.073402 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ddb6a-0ae7-4cb5-9d66-ca4ee440aea5 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:06.074282 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:06.114267 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19de5e-a7e9-4913-ba2b-e0b0d6c3126d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:06.115083 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19de5e-a7e9-4913-ba2b-e0b0d6c3126d None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:06.115314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19de5e-a7e9-4913-ba2b-e0b0d6c3126d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:06.115844 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19de5e-a7e9-4913-ba2b-e0b0d6c3126d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:06.125433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19de5e-a7e9-4913-ba2b-e0b0d6c3126d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:06.127486 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19de5e-a7e9-4913-ba2b-e0b0d6c3126d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:06.128560 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3336056-007b-4b36-8862-ee7900be6c98 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:06.144050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-df19de5e-a7e9-4913-ba2b-e0b0d6c3126d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:06.144050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:06.144050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:06.144050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:06.144050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:06.144050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:06.144050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:06.144050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:06.144050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:06.144050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:06.144050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:06.144050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:06.144050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:06:06.144050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:06:06.144050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:06:06.146076 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:06:06.146076 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:06.146076 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:06.146076 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:06.146076 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:06.146076 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:06.146076 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:06.146076 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:06.146076 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:06.146076 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:06.146076 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:06.146076 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:06.146076 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:06.146076 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:06.146076 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:06.146076 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:06.146076 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:06.147274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:06:06.147274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:06:06.147274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:06:06.147274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:06:06.147274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:06:06.147274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:06:06.147274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:06.147274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:06.147274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 304/602] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:06:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:06:06.161163 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19de5e-a7e9-4913-ba2b-e0b0d6c3126d None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:06.170574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718fa8b-4c07-46dc-9586-c4d6a731519a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:06.171278 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718fa8b-4c07-46dc-9586-c4d6a731519a None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:06.171705 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718fa8b-4c07-46dc-9586-c4d6a731519a None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:06.171981 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718fa8b-4c07-46dc-9586-c4d6a731519a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:06.180219 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718fa8b-4c07-46dc-9586-c4d6a731519a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:06.181864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718fa8b-4c07-46dc-9586-c4d6a731519a None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:06.195697 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-2718fa8b-4c07-46dc-9586-c4d6a731519a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:06.195697 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:06.195697 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:06.195697 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:06.195697 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:06.195697 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:06.195697 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:06.195697 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:06.195697 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:06.195697 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:06.195697 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:06.195697 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:06.195697 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:06:06.195697 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:06:06.195697 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:06:06.197020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:06:06.197020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:06.197020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:06.197020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:06.197020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:06.197020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:06.197020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:06.197020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:06.197020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:06.197020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:06.197020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:06.197020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:06.197020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:06.197020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:06.197020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:06.197020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:06.197020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:06.198099 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:06:06.198099 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:06:06.198099 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:06:06.198099 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:06:06.198099 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:06:06.198099 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:06:06.198099 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:06.198099 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:06.201887 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 299/603] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:06:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:06:06.203196 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718fa8b-4c07-46dc-9586-c4d6a731519a None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:06.364242 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718fa8b-4c07-46dc-9586-c4d6a731519a None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:06.383320 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:06.398080 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19de5e-a7e9-4913-ba2b-e0b0d6c3126d None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:06.414115 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49132ec7-9087-4c56-9c30-20b004ebba55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:06.415091 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49132ec7-9087-4c56-9c30-20b004ebba55 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:06.415950 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49132ec7-9087-4c56-9c30-20b004ebba55 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:06.416312 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49132ec7-9087-4c56-9c30-20b004ebba55 None admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:06.417154 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49132ec7-9087-4c56-9c30-20b004ebba55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:06.417832 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49132ec7-9087-4c56-9c30-20b004ebba55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:06.420065 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49132ec7-9087-4c56-9c30-20b004ebba55 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:06.437334 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:06.441031 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 300/604] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:06:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:06.442735 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645e21c-dd8d-4cb4-8fa1-5a17e40568fd None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:06.443322 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645e21c-dd8d-4cb4-8fa1-5a17e40568fd None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:06.443783 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645e21c-dd8d-4cb4-8fa1-5a17e40568fd None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:06.445578 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 301/605] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:06.446875 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645e21c-dd8d-4cb4-8fa1-5a17e40568fd None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:06.455389 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c64172-4809-4b2a-bfb4-eeaf808689fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:06.456043 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c64172-4809-4b2a-bfb4-eeaf808689fc None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:06.456325 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c64172-4809-4b2a-bfb4-eeaf808689fc None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:06.456599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c64172-4809-4b2a-bfb4-eeaf808689fc None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:06.457304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c64172-4809-4b2a-bfb4-eeaf808689fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:06.457618 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c64172-4809-4b2a-bfb4-eeaf808689fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:06.459309 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c64172-4809-4b2a-bfb4-eeaf808689fc None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:06.474502 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 305/606] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:06:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:06.475499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89fdbf-c7a3-4521-a3ad-bc8f4aeeb163 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:06.475868 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89fdbf-c7a3-4521-a3ad-bc8f4aeeb163 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:06.476140 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89fdbf-c7a3-4521-a3ad-bc8f4aeeb163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:06.496621 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-ef89fdbf-c7a3-4521-a3ad-bc8f4aeeb163 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:06.498567 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ef89fdbf-c7a3-4521-a3ad-bc8f4aeeb163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:06.585434 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 306/607] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:06:06.586648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef89fdbf-c7a3-4521-a3ad-bc8f4aeeb163 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:06.655817 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645e21c-dd8d-4cb4-8fa1-5a17e40568fd None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:06.712809 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef89fdbf-c7a3-4521-a3ad-bc8f4aeeb163 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:06.739779 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:06.741709 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eca249-bd70-4dc8-bd0d-f79c429e557c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:06.742389 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eca249-bd70-4dc8-bd0d-f79c429e557c None admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:06.742668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eca249-bd70-4dc8-bd0d-f79c429e557c None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:06.742943 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eca249-bd70-4dc8-bd0d-f79c429e557c None admin] PATH_INFO: `/v3/projects/fa3178684d184ec880f17640d19d55b7/users/e8a47a0d9bfd473cbf0147d7801442d0/roles/8cf764a0144a490a822fcf86d7aceeb0` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:06.770018 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6671a6-5e43-4b70-91dd-fab9e48a0f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:06.771857 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6671a6-5e43-4b70-91dd-fab9e48a0f43 None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:06.771857 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6671a6-5e43-4b70-91dd-fab9e48a0f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:06.772100 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6671a6-5e43-4b70-91dd-fab9e48a0f43 None admin] PATH_INFO: `/v3/projects/fa3178684d184ec880f17640d19d55b7/users/9f79599ed0b74bdbab198627ef72252a/roles/8cf764a0144a490a822fcf86d7aceeb0` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:06.777436 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eca249-bd70-4dc8-bd0d-f79c429e557c None admin] RBAC: Authorizing `identity:create_grant(project_id=fa3178684d184ec880f17640d19d55b7, user_id=e8a47a0d9bfd473cbf0147d7801442d0, role_id=8cf764a0144a490a822fcf86d7aceeb0)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:06.780647 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eca249-bd70-4dc8-bd0d-f79c429e557c None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:06.801746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6671a6-5e43-4b70-91dd-fab9e48a0f43 None admin] RBAC: Authorizing `identity:create_grant(project_id=fa3178684d184ec880f17640d19d55b7, user_id=9f79599ed0b74bdbab198627ef72252a, role_id=8cf764a0144a490a822fcf86d7aceeb0)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:06.803475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6671a6-5e43-4b70-91dd-fab9e48a0f43 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:06.804526 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 302/608] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Sep 5 12:06:06 2022] PUT /identity/v3/projects/fa3178684d184ec880f17640d19d55b7/users/e8a47a0d9bfd473cbf0147d7801442d0/roles/8cf764a0144a490a822fcf86d7aceeb0 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:06:06.823919 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 307/609] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Sep 5 12:06:06 2022] PUT /identity/v3/projects/fa3178684d184ec880f17640d19d55b7/users/9f79599ed0b74bdbab198627ef72252a/roles/8cf764a0144a490a822fcf86d7aceeb0 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:06:06.961503 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eca249-bd70-4dc8-bd0d-f79c429e557c None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:06.999585 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:07.110361 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eca249-bd70-4dc8-bd0d-f79c429e557c None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:07.150876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c509bb-e383-4020-bad7-eb4b86fc1715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:07.151902 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c509bb-e383-4020-bad7-eb4b86fc1715 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:07.152382 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c509bb-e383-4020-bad7-eb4b86fc1715 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:07.153148 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c509bb-e383-4020-bad7-eb4b86fc1715 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:07.168390 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c509bb-e383-4020-bad7-eb4b86fc1715 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:06:07.174129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-c5c509bb-e383-4020-bad7-eb4b86fc1715 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:06:07.174129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:07.174129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:07.174129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:07.174129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:07.174129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:07.174129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:07.174129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:07.174129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:07.174129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:07.174129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:07.174129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:07.174129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:06:07.174129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:06:07.174129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:06:07.174129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:06:07.175146 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:06:07.175146 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:06:07.175146 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:06:07.175146 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:06:07.175146 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:07.175146 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:07.175146 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:07.175146 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:07.175146 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:07.175146 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:07.175146 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:07.175146 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:07.175146 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:07.175146 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:07.175146 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:07.175146 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:07.176019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:07.176019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:07.176019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:07.176019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:07.176019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:06:07.176019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:06:07.176019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:06:07.176019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:06:07.176019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:06:07.176019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:06:07.176019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:06:07.176019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:07.183698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 303/610] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Sep 5 12:06:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:07.185516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6671a6-5e43-4b70-91dd-fab9e48a0f43 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:07.360223 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6671a6-5e43-4b70-91dd-fab9e48a0f43 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:07.399164 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e2c51-f99f-4ecb-b08d-4f94ac7d501b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:07.399947 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e2c51-f99f-4ecb-b08d-4f94ac7d501b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:07.400231 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e2c51-f99f-4ecb-b08d-4f94ac7d501b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:07.400506 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e2c51-f99f-4ecb-b08d-4f94ac7d501b None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:07.401180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e2c51-f99f-4ecb-b08d-4f94ac7d501b None admin] RBAC: Adding query filter params (name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:07.401503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e2c51-f99f-4ecb-b08d-4f94ac7d501b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:07.403252 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e2c51-f99f-4ecb-b08d-4f94ac7d501b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:07.412834 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 308/611] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Sep 5 12:06:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:06:07.420907 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c509bb-e383-4020-bad7-eb4b86fc1715 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:07.542998 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:07.572323 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c509bb-e383-4020-bad7-eb4b86fc1715 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:07.646251 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d216b023-cc8f-4aba-a37f-ee8695f846bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:07.647416 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216b023-cc8f-4aba-a37f-ee8695f846bf None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:07.648173 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216b023-cc8f-4aba-a37f-ee8695f846bf None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:07.648884 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216b023-cc8f-4aba-a37f-ee8695f846bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:07.666189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d216b023-cc8f-4aba-a37f-ee8695f846bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:07.668233 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d216b023-cc8f-4aba-a37f-ee8695f846bf None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:07.692603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-d216b023-cc8f-4aba-a37f-ee8695f846bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:07.692603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:07.692603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:07.692603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:07.692603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:07.692603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:07.692603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:07.692603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:07.692603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:07.692603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:07.692603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:07.692603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:07.692603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:07.692603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:07.692603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:07.692603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:07.693811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:07.693811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:07.693811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:07.693811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:07.693811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:07.693811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:07.693811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:07.693811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:07.693811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:07.693811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:07.693811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:07.693811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:07.693811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:07.693811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:07.693811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:07.693811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:07.693811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:07.694738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:07.694738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:07.694738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:07.696535 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 304/612] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:07.703592 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e2c51-f99f-4ecb-b08d-4f94ac7d501b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:07.762247 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:07.870416 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e2c51-f99f-4ecb-b08d-4f94ac7d501b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:07.928223 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2afe77b-7474-46fe-b486-8c995b4e1242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:07.928814 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afe77b-7474-46fe-b486-8c995b4e1242 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:07.929096 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afe77b-7474-46fe-b486-8c995b4e1242 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:07.929370 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afe77b-7474-46fe-b486-8c995b4e1242 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:07.930183 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2afe77b-7474-46fe-b486-8c995b4e1242 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:07.930559 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2afe77b-7474-46fe-b486-8c995b4e1242 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:07.932181 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2afe77b-7474-46fe-b486-8c995b4e1242 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:07.957383 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 309/613] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:07.963936 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d216b023-cc8f-4aba-a37f-ee8695f846bf None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:08.118077 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba6278-3429-4260-b77a-cf812e134569 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:08.118432 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba6278-3429-4260-b77a-cf812e134569 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:08.118630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba6278-3429-4260-b77a-cf812e134569 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:08.119109 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d216b023-cc8f-4aba-a37f-ee8695f846bf None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:08.121433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 310/614] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:08.126570 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061d788-c142-407e-b035-4f001277cda6 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:08.126888 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061d788-c142-407e-b035-4f001277cda6 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:08.127109 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061d788-c142-407e-b035-4f001277cda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:08.167295 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdfb00c-a339-451f-bd6f-495c9628f4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:08.168007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdfb00c-a339-451f-bd6f-495c9628f4c9 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:08.168290 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdfb00c-a339-451f-bd6f-495c9628f4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:08.168567 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdfb00c-a339-451f-bd6f-495c9628f4c9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:08.170401 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-a061d788-c142-407e-b035-4f001277cda6 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:08.171698 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a061d788-c142-407e-b035-4f001277cda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:08.175995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:08.191641 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdfb00c-a339-451f-bd6f-495c9628f4c9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:08.194235 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdfb00c-a339-451f-bd6f-495c9628f4c9 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:08.217791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-ebdfb00c-a339-451f-bd6f-495c9628f4c9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 05 12:06:08.217791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:08.217791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:08.217791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:08.217791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:08.217791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:08.217791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:08.217791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:08.217791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:08.217791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:08.217791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:08.217791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:08.217791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:06:08.217791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:06:08.217791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:06:08.217791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:06:08.218820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:08.218820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:08.218820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:06:08.218820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:08.218820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:06:08.218820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:08.218820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:08.218820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:08.218820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:08.218820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:08.218820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:08.218820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:08.218820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:08.218820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:08.218820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:08.218820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:08.218820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:08.218820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:08.219823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:08.219823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:08.219823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:06:08.219823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:06:08.219823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:06:08.219823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:06:08.219823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:06:08.219823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:06:08.219823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 05 12:06:08.219823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:08.222523 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 305/615] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:06:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:08.228692 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdfb00c-a339-451f-bd6f-495c9628f4c9 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:08.293180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 311/616] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:06:08.399742 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdfb00c-a339-451f-bd6f-495c9628f4c9 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:08.442751 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5831da-f31a-40bf-8392-8bc89f8e30d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:08.444345 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5831da-f31a-40bf-8392-8bc89f8e30d3 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:08.444883 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5831da-f31a-40bf-8392-8bc89f8e30d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:08.445219 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917660e0-c20a-425e-b895-f7f1c0b5afba None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:08.445509 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917660e0-c20a-425e-b895-f7f1c0b5afba None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:08.445781 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5831da-f31a-40bf-8392-8bc89f8e30d3 None admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:08.445886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917660e0-c20a-425e-b895-f7f1c0b5afba None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:08.446831 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5831da-f31a-40bf-8392-8bc89f8e30d3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:08.447346 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5831da-f31a-40bf-8392-8bc89f8e30d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:08.447470 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 312/617] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:08.449228 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5831da-f31a-40bf-8392-8bc89f8e30d3 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:08.453822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399bfe30-d95a-49ef-a8de-f1e5a8e5addf None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:08.454148 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399bfe30-d95a-49ef-a8de-f1e5a8e5addf None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:08.454423 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399bfe30-d95a-49ef-a8de-f1e5a8e5addf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:08.480961 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 306/618] 10.222.0.16 () {56 vars in 1135 bytes} [Mon Sep 5 12:06:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:08.487781 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5831da-f31a-40bf-8392-8bc89f8e30d3 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:08.496514 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-399bfe30-d95a-49ef-a8de-f1e5a8e5addf None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:08.500339 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-399bfe30-d95a-49ef-a8de-f1e5a8e5addf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:08.539903 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:08.595817 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:08.641041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 313/619] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:06:08.674849 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5831da-f31a-40bf-8392-8bc89f8e30d3 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:08.727767 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0cac6b-4a4d-446a-8fc3-78c2cfc8501a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:08.728446 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0cac6b-4a4d-446a-8fc3-78c2cfc8501a None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:08.728730 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0cac6b-4a4d-446a-8fc3-78c2cfc8501a None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:08.729011 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0cac6b-4a4d-446a-8fc3-78c2cfc8501a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:08.748286 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0cac6b-4a4d-446a-8fc3-78c2cfc8501a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:08.750967 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0cac6b-4a4d-446a-8fc3-78c2cfc8501a None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:08.780447 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-ec0cac6b-4a4d-446a-8fc3-78c2cfc8501a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:08.780447 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:08.780447 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:08.780447 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:08.780447 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:08.780447 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:08.780447 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:08.780447 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:08.780447 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:08.780447 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:08.780447 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:08.780447 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:08.780447 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:08.780447 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:08.780447 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:08.780447 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:08.781463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:08.781463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:08.781463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:08.781463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:08.781463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:08.781463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:08.781463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:08.781463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:08.781463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:08.781463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:08.781463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:08.781463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:08.781463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:08.781463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:08.781463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:08.781463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:08.781463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:08.782603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:08.782603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:08.782603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:08.782603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 307/620] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:08.789350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399bfe30-d95a-49ef-a8de-f1e5a8e5addf None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:08.825597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0cac6b-4a4d-446a-8fc3-78c2cfc8501a None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:08.924758 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399bfe30-d95a-49ef-a8de-f1e5a8e5addf None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:08.969409 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0841c3d-5550-498f-b771-5d6bf6770aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:08.970034 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0841c3d-5550-498f-b771-5d6bf6770aba None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:08.970316 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0841c3d-5550-498f-b771-5d6bf6770aba None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:08.970590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0841c3d-5550-498f-b771-5d6bf6770aba None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:08.971248 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0841c3d-5550-498f-b771-5d6bf6770aba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:08.971573 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0841c3d-5550-498f-b771-5d6bf6770aba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:08.972963 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0841c3d-5550-498f-b771-5d6bf6770aba None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:09.020994 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 314/621] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:09.029457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0841c3d-5550-498f-b771-5d6bf6770aba None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:09.055063 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:09.055608 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0cac6b-4a4d-446a-8fc3-78c2cfc8501a None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:09.060614 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:09.109123 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af46cf-3b35-40fb-9581-88645d21ce81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:09.109842 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af46cf-3b35-40fb-9581-88645d21ce81 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:09.110125 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af46cf-3b35-40fb-9581-88645d21ce81 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:09.110395 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af46cf-3b35-40fb-9581-88645d21ce81 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:09.122102 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77af46cf-3b35-40fb-9581-88645d21ce81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:06:09.127956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-77af46cf-3b35-40fb-9581-88645d21ce81 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:06:09.127956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:09.127956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:09.127956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:09.127956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:09.127956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:09.127956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:09.127956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:09.127956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:09.127956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:09.127956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:09.127956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:09.127956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:06:09.127956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:06:09.127956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:06:09.127956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:06:09.128930 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:06:09.128930 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:06:09.128930 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:06:09.128930 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:06:09.128930 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:09.128930 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:09.128930 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:09.128930 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:09.128930 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:09.128930 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:09.128930 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:09.128930 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:09.128930 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:09.128930 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:09.128930 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:09.128930 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:09.130181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:09.130181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:09.130181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:09.130181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:09.130181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:06:09.130181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:06:09.130181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:06:09.130181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:06:09.130181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:06:09.130181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:06:09.130181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:06:09.130181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:09.130181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 308/622] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Sep 5 12:06:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:09.130181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af46cf-3b35-40fb-9581-88645d21ce81 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:09.180870 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0841c3d-5550-498f-b771-5d6bf6770aba None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:09.228236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1c799-9ddf-47b4-a6fd-c82063578588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:09.229135 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1c799-9ddf-47b4-a6fd-c82063578588 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:09.229474 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1c799-9ddf-47b4-a6fd-c82063578588 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:09.230127 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1c799-9ddf-47b4-a6fd-c82063578588 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:09.241837 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af46cf-3b35-40fb-9581-88645d21ce81 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:09.250250 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e1c799-9ddf-47b4-a6fd-c82063578588 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:09.251772 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e1c799-9ddf-47b4-a6fd-c82063578588 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:09.266987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-12e1c799-9ddf-47b4-a6fd-c82063578588 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:09.266987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:09.266987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:09.266987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:09.266987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:09.266987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:09.266987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:09.266987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:09.266987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:09.266987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:09.266987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:09.266987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:09.266987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:06:09.266987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:06:09.266987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:06:09.268023 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:06:09.268023 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:09.268023 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:09.268023 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:09.268023 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:09.268023 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:09.268023 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:09.268023 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:09.268023 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:09.268023 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:09.268023 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:09.268023 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:09.268023 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:09.268023 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:09.268023 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:09.268023 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:09.268023 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:09.268936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:06:09.268936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:06:09.268936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:06:09.268936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:06:09.268936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:06:09.268936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:06:09.268936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:09.268936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:09.273418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 315/623] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:06:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:09.274364 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1c799-9ddf-47b4-a6fd-c82063578588 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:09.289719 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a367f-1515-4877-819f-0915fce4d1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:09.291184 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a367f-1515-4877-819f-0915fce4d1d0 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:09.291852 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a367f-1515-4877-819f-0915fce4d1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:09.292431 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a367f-1515-4877-819f-0915fce4d1d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:09.307911 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96a367f-1515-4877-819f-0915fce4d1d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:06:09.315605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-f96a367f-1515-4877-819f-0915fce4d1d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:06:09.315605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:09.315605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:09.315605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:09.315605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:09.315605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:09.315605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:09.315605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:09.315605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:09.315605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:09.315605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:09.315605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:09.315605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:06:09.315605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:06:09.315605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:06:09.315605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:06:09.316576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:06:09.316576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:06:09.316576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:06:09.316576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:06:09.316576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:09.316576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:09.316576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:09.316576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:09.316576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:09.316576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:09.316576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:09.316576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:09.316576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:09.316576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:09.316576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:09.316576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:09.317415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:09.317415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:09.317415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:09.317415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:09.317415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:06:09.317415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:06:09.317415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:06:09.317415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:06:09.317415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:06:09.317415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:06:09.317415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:06:09.317415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:09.320933 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 309/624] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Sep 5 12:06:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:09.321983 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a367f-1515-4877-819f-0915fce4d1d0 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:09.368229 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:09.378469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:09.415720 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1c799-9ddf-47b4-a6fd-c82063578588 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:09.454899 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973cc15-730f-4489-8f4d-723c8481e381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:09.456242 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973cc15-730f-4489-8f4d-723c8481e381 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:09.456531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973cc15-730f-4489-8f4d-723c8481e381 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:09.456809 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973cc15-730f-4489-8f4d-723c8481e381 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:09.457507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973cc15-730f-4489-8f4d-723c8481e381 None admin] RBAC: Adding query filter params (name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:09.458281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973cc15-730f-4489-8f4d-723c8481e381 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:09.460157 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973cc15-730f-4489-8f4d-723c8481e381 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:09.475503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 316/625] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Sep 5 12:06:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:09.475503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973cc15-730f-4489-8f4d-723c8481e381 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:09.491618 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a367f-1515-4877-819f-0915fce4d1d0 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:09.532346 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4770ae-bc80-435a-99a7-42a8b5914e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:09.533039 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4770ae-bc80-435a-99a7-42a8b5914e50 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:09.533319 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4770ae-bc80-435a-99a7-42a8b5914e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:09.533635 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4770ae-bc80-435a-99a7-42a8b5914e50 None admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:09.534516 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4770ae-bc80-435a-99a7-42a8b5914e50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:09.534827 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4770ae-bc80-435a-99a7-42a8b5914e50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:09.536342 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4770ae-bc80-435a-99a7-42a8b5914e50 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:09.556771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 310/626] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:06:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:09.556771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bfa200-e103-4a08-ad95-78b3fe9cfff9 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:09.556771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bfa200-e103-4a08-ad95-78b3fe9cfff9 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:09.556771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bfa200-e103-4a08-ad95-78b3fe9cfff9 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:09.556771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 311/627] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:09.556771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bfa200-e103-4a08-ad95-78b3fe9cfff9 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:09.589975 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973cc15-730f-4489-8f4d-723c8481e381 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:09.626414 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aef6a5-5566-41de-8aca-4935b75d46e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:09.627027 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aef6a5-5566-41de-8aca-4935b75d46e1 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:09.627311 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aef6a5-5566-41de-8aca-4935b75d46e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:09.627610 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aef6a5-5566-41de-8aca-4935b75d46e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:09.628287 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aef6a5-5566-41de-8aca-4935b75d46e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:09.628591 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aef6a5-5566-41de-8aca-4935b75d46e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:09.629993 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aef6a5-5566-41de-8aca-4935b75d46e1 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:09.631756 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:09.640524 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 317/628] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Sep 5 12:06:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:09.641801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b441432c-8589-45aa-b703-afd7b34f2364 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:09.642081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b441432c-8589-45aa-b703-afd7b34f2364 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:09.642352 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b441432c-8589-45aa-b703-afd7b34f2364 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:09.644200 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 318/629] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:09.644654 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b441432c-8589-45aa-b703-afd7b34f2364 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:09.660280 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:09.683363 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bfa200-e103-4a08-ad95-78b3fe9cfff9 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:09.722456 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08cc79c-c983-435f-805a-8d59acfe2c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:09.723073 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08cc79c-c983-435f-805a-8d59acfe2c58 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:09.723351 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08cc79c-c983-435f-805a-8d59acfe2c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:09.723655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08cc79c-c983-435f-805a-8d59acfe2c58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:09.734172 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08cc79c-c983-435f-805a-8d59acfe2c58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:09.735670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08cc79c-c983-435f-805a-8d59acfe2c58 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:09.752670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-e08cc79c-c983-435f-805a-8d59acfe2c58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:09.752670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:09.752670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:09.752670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:09.752670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:09.752670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:09.752670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:09.752670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:09.752670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:09.752670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:09.752670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:09.752670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:09.752670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:09.752670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:09.752670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:09.752670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:09.753895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:09.753895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:09.753895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:09.753895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:09.753895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:09.753895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:09.753895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:09.753895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:09.753895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:09.753895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:09.753895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:09.753895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:09.753895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:09.753895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:09.753895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:09.753895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:09.753895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:09.754814 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:09.754814 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:09.754814 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:09.754814 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 312/630] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:09.755046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa55f2-093d-4783-8dce-d1b06f3d629b None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:09.755294 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa55f2-093d-4783-8dce-d1b06f3d629b None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:09.755563 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa55f2-093d-4783-8dce-d1b06f3d629b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:09.761596 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b441432c-8589-45aa-b703-afd7b34f2364 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:09.779356 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-d3fa55f2-093d-4783-8dce-d1b06f3d629b None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:09.781316 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-d3fa55f2-093d-4783-8dce-d1b06f3d629b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:09.810393 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc317d1-4570-4b06-a63c-b128b481288c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:09.810957 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc317d1-4570-4b06-a63c-b128b481288c None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:09.811234 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc317d1-4570-4b06-a63c-b128b481288c None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:09.811508 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc317d1-4570-4b06-a63c-b128b481288c None admin] PATH_INFO: `/v3/projects/fa3178684d184ec880f17640d19d55b7/users/857eb7657e7d45938baca17f91a16901/roles/8cf764a0144a490a822fcf86d7aceeb0` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:09.826779 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc317d1-4570-4b06-a63c-b128b481288c None admin] RBAC: Authorizing `identity:create_grant(project_id=fa3178684d184ec880f17640d19d55b7, user_id=857eb7657e7d45938baca17f91a16901, role_id=8cf764a0144a490a822fcf86d7aceeb0)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:09.828245 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc317d1-4570-4b06-a63c-b128b481288c None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:09.841494 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 319/631] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Sep 5 12:06:09 2022] PUT /identity/v3/projects/fa3178684d184ec880f17640d19d55b7/users/857eb7657e7d45938baca17f91a16901/roles/8cf764a0144a490a822fcf86d7aceeb0 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:06:09.848644 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc317d1-4570-4b06-a63c-b128b481288c None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:09.873148 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 313/632] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:06:09.875237 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941f3f3-f82d-4b19-bc0b-17aa990256be None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:09.875888 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941f3f3-f82d-4b19-bc0b-17aa990256be None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:09.877248 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941f3f3-f82d-4b19-bc0b-17aa990256be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:09.905915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-2941f3f3-f82d-4b19-bc0b-17aa990256be None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:09.907038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:09.909101 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-2941f3f3-f82d-4b19-bc0b-17aa990256be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:09.982390 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:09.992115 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 314/633] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:06:09.993289 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941f3f3-f82d-4b19-bc0b-17aa990256be None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:09.996905 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc317d1-4570-4b06-a63c-b128b481288c None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:10.044844 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac58e18-0e98-46ac-bdcc-146a09919ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:10.045502 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac58e18-0e98-46ac-bdcc-146a09919ccb None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:10.045791 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac58e18-0e98-46ac-bdcc-146a09919ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:10.046072 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac58e18-0e98-46ac-bdcc-146a09919ccb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:10.063153 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac58e18-0e98-46ac-bdcc-146a09919ccb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:10.065253 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac58e18-0e98-46ac-bdcc-146a09919ccb None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:10.092818 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-8ac58e18-0e98-46ac-bdcc-146a09919ccb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:10.092818 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:10.092818 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:10.092818 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:10.092818 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:10.092818 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:10.092818 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:10.092818 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:10.092818 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:10.092818 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:10.092818 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:10.092818 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:10.092818 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:10.092818 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:10.092818 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:10.092818 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:10.093858 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:10.093858 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:10.093858 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:10.093858 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:10.093858 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:10.093858 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:10.093858 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:10.093858 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:10.093858 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:10.093858 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:10.093858 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:10.093858 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:10.093858 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:10.093858 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:10.093858 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:10.093858 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:10.093858 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:10.094746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:10.094746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:10.094746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:10.094746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 320/634] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:10.099850 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac58e18-0e98-46ac-bdcc-146a09919ccb None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:10.151101 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941f3f3-f82d-4b19-bc0b-17aa990256be None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:10.188752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70169641-1764-4d30-b788-7cf4c256a941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:10.189359 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70169641-1764-4d30-b788-7cf4c256a941 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:10.189637 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70169641-1764-4d30-b788-7cf4c256a941 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:10.189906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70169641-1764-4d30-b788-7cf4c256a941 None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:10.190551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70169641-1764-4d30-b788-7cf4c256a941 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:10.190849 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70169641-1764-4d30-b788-7cf4c256a941 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:10.192245 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70169641-1764-4d30-b788-7cf4c256a941 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:10.204815 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 315/635] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:10.205467 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70169641-1764-4d30-b788-7cf4c256a941 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:10.226033 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:10.232752 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac58e18-0e98-46ac-bdcc-146a09919ccb None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:10.273260 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:10.286843 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc19755e-aae4-40d6-bddd-ba6740af9a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:10.287722 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc19755e-aae4-40d6-bddd-ba6740af9a94 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:10.288098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc19755e-aae4-40d6-bddd-ba6740af9a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:10.288975 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc19755e-aae4-40d6-bddd-ba6740af9a94 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:10.289724 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc19755e-aae4-40d6-bddd-ba6740af9a94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:10.290384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc19755e-aae4-40d6-bddd-ba6740af9a94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:10.292041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc19755e-aae4-40d6-bddd-ba6740af9a94 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:10.307488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 321/636] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:10.308849 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc19755e-aae4-40d6-bddd-ba6740af9a94 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:10.327391 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70169641-1764-4d30-b788-7cf4c256a941 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:10.362461 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ade32-7ff3-4a3a-a711-7acd12e3585f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:10.363090 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ade32-7ff3-4a3a-a711-7acd12e3585f None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:10.363372 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ade32-7ff3-4a3a-a711-7acd12e3585f None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:10.363675 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ade32-7ff3-4a3a-a711-7acd12e3585f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:10.378294 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ade32-7ff3-4a3a-a711-7acd12e3585f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:06:10.383583 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-692ade32-7ff3-4a3a-a711-7acd12e3585f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:06:10.383583 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:10.383583 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:10.383583 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:10.383583 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:10.383583 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:10.383583 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:10.383583 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:10.383583 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:10.383583 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:10.383583 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:10.383583 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:10.383583 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:06:10.383583 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:06:10.383583 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:06:10.383583 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:06:10.384540 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:06:10.384540 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:06:10.384540 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:06:10.384540 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:06:10.384540 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:10.384540 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:10.384540 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:10.384540 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:10.384540 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:10.384540 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:10.384540 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:10.384540 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:10.384540 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:10.384540 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:10.384540 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:10.384540 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:10.385412 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:10.385412 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:10.385412 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:10.385412 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:10.385412 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:06:10.385412 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:06:10.385412 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:06:10.385412 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:06:10.385412 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:06:10.385412 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:06:10.385412 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:06:10.385412 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:10.389639 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 316/637] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Sep 5 12:06:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:10.390327 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ade32-7ff3-4a3a-a711-7acd12e3585f None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:10.425875 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc19755e-aae4-40d6-bddd-ba6740af9a94 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:10.467229 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1131f94b-2af4-4654-a63e-1dde7c47bd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:10.467914 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131f94b-2af4-4654-a63e-1dde7c47bd6b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:10.468193 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131f94b-2af4-4654-a63e-1dde7c47bd6b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:10.468462 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131f94b-2af4-4654-a63e-1dde7c47bd6b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:10.477184 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1131f94b-2af4-4654-a63e-1dde7c47bd6b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:10.478639 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1131f94b-2af4-4654-a63e-1dde7c47bd6b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:10.492430 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-1131f94b-2af4-4654-a63e-1dde7c47bd6b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 05 12:06:10.492430 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:10.492430 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:10.492430 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:10.492430 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:10.492430 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:10.492430 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:10.492430 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:10.492430 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:10.492430 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:10.492430 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:10.492430 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:10.492430 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:06:10.492430 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:06:10.492430 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:06:10.492430 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:06:10.493499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:10.493499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:10.493499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:06:10.493499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:10.493499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:06:10.493499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:10.493499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:10.493499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:10.493499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:10.493499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:10.493499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:10.493499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:10.493499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:10.493499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:10.493499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:10.493499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:10.493499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:10.493499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:10.494480 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:10.494480 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:10.494480 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:06:10.494480 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:06:10.494480 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:06:10.494480 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:06:10.494480 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:06:10.494480 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:06:10.494480 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 05 12:06:10.494480 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:10.498484 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 322/638] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:06:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:10.499595 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1131f94b-2af4-4654-a63e-1dde7c47bd6b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:10.504724 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ade32-7ff3-4a3a-a711-7acd12e3585f None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:10.520579 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:10.529536 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:10.552157 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8f1ca-e778-4951-ab7a-4d50aac2f8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:10.552802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8f1ca-e778-4951-ab7a-4d50aac2f8a4 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:10.553086 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8f1ca-e778-4951-ab7a-4d50aac2f8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:10.553364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8f1ca-e778-4951-ab7a-4d50aac2f8a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:10.564290 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f8f1ca-e778-4951-ab7a-4d50aac2f8a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:06:10.570919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-17f8f1ca-e778-4951-ab7a-4d50aac2f8a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:06:10.570919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:10.570919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:10.570919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:10.570919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:10.570919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:10.570919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:10.570919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:10.570919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:10.570919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:10.570919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:10.570919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:10.570919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:06:10.570919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:06:10.570919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:06:10.570919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:06:10.572091 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:06:10.572091 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:06:10.572091 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:06:10.572091 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:06:10.572091 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:10.572091 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:10.572091 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:10.572091 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:10.572091 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:10.572091 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:10.572091 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:10.572091 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:10.572091 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:10.572091 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:10.572091 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:10.572091 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:10.573170 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:10.573170 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:10.573170 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:10.573170 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:10.573170 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:06:10.573170 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:06:10.573170 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:06:10.573170 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:06:10.573170 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:06:10.573170 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:06:10.573170 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:06:10.573170 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:10.574919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 317/639] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Sep 5 12:06:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:10.575676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8f1ca-e778-4951-ab7a-4d50aac2f8a4 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:10.616290 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1131f94b-2af4-4654-a63e-1dde7c47bd6b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:10.651693 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf10a07-5760-4c51-aca9-8eaa69076500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:10.652322 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf10a07-5760-4c51-aca9-8eaa69076500 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:10.652847 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf10a07-5760-4c51-aca9-8eaa69076500 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:10.653121 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf10a07-5760-4c51-aca9-8eaa69076500 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:10.661498 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf10a07-5760-4c51-aca9-8eaa69076500 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:10.662931 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf10a07-5760-4c51-aca9-8eaa69076500 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:10.674149 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-acf10a07-5760-4c51-aca9-8eaa69076500 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:06:10.674149 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:10.674149 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:10.674149 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:10.674149 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:10.674149 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:10.674149 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:10.674149 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:10.674149 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:10.674149 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:10.674149 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:10.674149 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:10.674149 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:06:10.674149 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:06:10.674149 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:06:10.674149 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:06:10.675198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:10.675198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:10.675198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:06:10.675198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:10.675198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:06:10.675198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:10.675198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:10.675198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:10.675198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:10.675198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:10.675198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:10.675198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:10.675198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:10.675198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:10.675198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:10.675198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:10.675198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:10.675198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:10.676235 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:10.676235 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:10.676235 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:06:10.676235 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:06:10.676235 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:06:10.676235 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:06:10.676235 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:06:10.676235 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:06:10.676235 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:06:10.676235 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:10.679993 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 323/640] 10.222.0.16 () {56 vars in 1093 bytes} [Mon Sep 5 12:06:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:10.680634 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf10a07-5760-4c51-aca9-8eaa69076500 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:10.692579 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8f1ca-e778-4951-ab7a-4d50aac2f8a4 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:10.727097 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09ab5e-0509-4830-be8e-f175f45d5c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:10.727751 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09ab5e-0509-4830-be8e-f175f45d5c19 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:10.728030 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09ab5e-0509-4830-be8e-f175f45d5c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:10.728298 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09ab5e-0509-4830-be8e-f175f45d5c19 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:10.728957 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a09ab5e-0509-4830-be8e-f175f45d5c19 None admin] RBAC: Adding query filter params (name=service) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:10.729256 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a09ab5e-0509-4830-be8e-f175f45d5c19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:10.730619 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a09ab5e-0509-4830-be8e-f175f45d5c19 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:10.738851 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 318/641] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Sep 5 12:06:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:10.739825 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09ab5e-0509-4830-be8e-f175f45d5c19 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:10.771142 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:10.784317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:10.795804 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf10a07-5760-4c51-aca9-8eaa69076500 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:10.832202 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce952b-00ad-4a37-8e21-38da8929c10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:10.832806 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce952b-00ad-4a37-8e21-38da8929c10f None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:10.833085 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce952b-00ad-4a37-8e21-38da8929c10f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:10.833350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce952b-00ad-4a37-8e21-38da8929c10f None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:10.834072 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce952b-00ad-4a37-8e21-38da8929c10f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:10.834405 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce952b-00ad-4a37-8e21-38da8929c10f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:10.835878 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce952b-00ad-4a37-8e21-38da8929c10f None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:10.854043 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 324/642] 10.222.0.16 () {56 vars in 1137 bytes} [Mon Sep 5 12:06:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:10.854407 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09ab5e-0509-4830-be8e-f175f45d5c19 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:10.854868 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce952b-00ad-4a37-8e21-38da8929c10f None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:10.893279 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a4f164-2625-4276-b595-e8b20eb96e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:10.893805 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4f164-2625-4276-b595-e8b20eb96e37 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:10.894084 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4f164-2625-4276-b595-e8b20eb96e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:10.894360 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4f164-2625-4276-b595-e8b20eb96e37 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:10.895032 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a4f164-2625-4276-b595-e8b20eb96e37 None admin] RBAC: Adding query filter params (name=service) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:10.895335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a4f164-2625-4276-b595-e8b20eb96e37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:10.896734 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a4f164-2625-4276-b595-e8b20eb96e37 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:10.905951 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 319/643] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Sep 5 12:06:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:10.908736 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a4f164-2625-4276-b595-e8b20eb96e37 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:10.961014 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce952b-00ad-4a37-8e21-38da8929c10f None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:10.996094 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a72e5d-ee45-494e-97f3-68bd556ec265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:10.996702 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a72e5d-ee45-494e-97f3-68bd556ec265 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:10.996963 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a72e5d-ee45-494e-97f3-68bd556ec265 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:10.997237 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a72e5d-ee45-494e-97f3-68bd556ec265 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:10.997987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a72e5d-ee45-494e-97f3-68bd556ec265 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:10.998286 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a72e5d-ee45-494e-97f3-68bd556ec265 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:10.999735 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a72e5d-ee45-494e-97f3-68bd556ec265 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:11.002334 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:11.017670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:11.018935 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 325/644] 10.222.0.16 () {56 vars in 1133 bytes} [Mon Sep 5 12:06:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:11.020552 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a72e5d-ee45-494e-97f3-68bd556ec265 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:11.021234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a4f164-2625-4276-b595-e8b20eb96e37 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:11.062828 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c942e40-76ad-4784-93fa-d65f0e5fb020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:11.063873 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c942e40-76ad-4784-93fa-d65f0e5fb020 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:11.064419 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c942e40-76ad-4784-93fa-d65f0e5fb020 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:11.064872 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c942e40-76ad-4784-93fa-d65f0e5fb020 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:11.077365 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c942e40-76ad-4784-93fa-d65f0e5fb020 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:11.078919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c942e40-76ad-4784-93fa-d65f0e5fb020 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:11.093733 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-8c942e40-76ad-4784-93fa-d65f0e5fb020 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:11.093733 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:11.093733 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:11.093733 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:11.093733 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:11.093733 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:11.093733 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:11.093733 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:11.093733 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:11.093733 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:11.093733 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:11.093733 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:11.093733 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:11.093733 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:11.093733 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:11.093733 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:11.094762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:11.094762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:11.094762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:11.094762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:11.094762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:11.094762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:11.094762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:11.094762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:11.094762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:11.094762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:11.094762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:11.094762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:11.094762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:11.094762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:11.094762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:11.094762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:11.094762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:11.096478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:11.096478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:11.096478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:11.096478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 320/645] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:11.096478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c942e40-76ad-4784-93fa-d65f0e5fb020 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:11.132870 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a72e5d-ee45-494e-97f3-68bd556ec265 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:11.170461 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaecb60-42b5-4f24-9dd4-af15dd1c8fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:11.170977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaecb60-42b5-4f24-9dd4-af15dd1c8fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:11.171259 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaecb60-42b5-4f24-9dd4-af15dd1c8fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:11.171553 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaecb60-42b5-4f24-9dd4-af15dd1c8fd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:11.210192 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c942e40-76ad-4784-93fa-d65f0e5fb020 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:11.241341 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:11.246948 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723c187-a07a-4aeb-b436-72f3e7006fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:11.248160 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723c187-a07a-4aeb-b436-72f3e7006fea None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:11.248442 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723c187-a07a-4aeb-b436-72f3e7006fea None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:11.248716 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723c187-a07a-4aeb-b436-72f3e7006fea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:11.253721 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaecb60-42b5-4f24-9dd4-af15dd1c8fd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:11.256032 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaecb60-42b5-4f24-9dd4-af15dd1c8fd3 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:11.261235 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4723c187-a07a-4aeb-b436-72f3e7006fea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:11.262767 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4723c187-a07a-4aeb-b436-72f3e7006fea None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:11.278959 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-4723c187-a07a-4aeb-b436-72f3e7006fea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:11.278959 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:11.278959 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:11.278959 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:11.278959 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:11.278959 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:11.278959 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:11.278959 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:11.278959 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:11.278959 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:11.278959 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:11.278959 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:11.278959 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:11.278959 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:11.278959 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:11.278959 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:11.280048 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:11.280048 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:11.280048 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:11.280048 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:11.280048 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:11.280048 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:11.280048 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:11.280048 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:11.280048 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:11.280048 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:11.280048 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:11.280048 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:11.280048 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:11.280048 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:11.280048 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:11.280048 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:11.280048 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:11.280954 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:11.280954 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:11.280954 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:11.280954 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 321/646] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:11.280954 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723c187-a07a-4aeb-b436-72f3e7006fea None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:11.286242 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:11.380004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 326/647] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:06:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:06:11.380790 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaecb60-42b5-4f24-9dd4-af15dd1c8fd3 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:11.405807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723c187-a07a-4aeb-b436-72f3e7006fea None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:11.445030 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c3778-09e0-4086-a468-1f70e07d492c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:11.445546 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c3778-09e0-4086-a468-1f70e07d492c None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:11.445819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c3778-09e0-4086-a468-1f70e07d492c None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:11.446078 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c3778-09e0-4086-a468-1f70e07d492c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:11.483171 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:11.490474 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaecb60-42b5-4f24-9dd4-af15dd1c8fd3 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:11.530984 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428c3778-09e0-4086-a468-1f70e07d492c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:11.532481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428c3778-09e0-4086-a468-1f70e07d492c None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:11.532887 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b3bc5-dd82-48d7-af00-787c4f28f63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:11.533503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b3bc5-dd82-48d7-af00-787c4f28f63a None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:11.533783 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b3bc5-dd82-48d7-af00-787c4f28f63a None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:11.534055 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b3bc5-dd82-48d7-af00-787c4f28f63a None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:11.534694 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86b3bc5-dd82-48d7-af00-787c4f28f63a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:11.534998 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86b3bc5-dd82-48d7-af00-787c4f28f63a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:11.536321 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:11.536430 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86b3bc5-dd82-48d7-af00-787c4f28f63a None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:11.553511 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 327/648] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:11.553511 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b3bc5-dd82-48d7-af00-787c4f28f63a None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:11.648601 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 322/649] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:06:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:06:11.650023 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c3778-09e0-4086-a468-1f70e07d492c None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:11.656246 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b3bc5-dd82-48d7-af00-787c4f28f63a None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:11.692732 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3592199-d1b2-42c2-90f2-6897bd59d720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:11.693384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3592199-d1b2-42c2-90f2-6897bd59d720 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:11.693669 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3592199-d1b2-42c2-90f2-6897bd59d720 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:11.693942 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3592199-d1b2-42c2-90f2-6897bd59d720 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:11.694592 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3592199-d1b2-42c2-90f2-6897bd59d720 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:11.694895 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3592199-d1b2-42c2-90f2-6897bd59d720 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:11.696340 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3592199-d1b2-42c2-90f2-6897bd59d720 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:11.708627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 328/650] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:11.709470 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3592199-d1b2-42c2-90f2-6897bd59d720 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:11.725428 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:11.780476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c3778-09e0-4086-a468-1f70e07d492c None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:11.783831 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:11.818758 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7b1a4-0c95-41fc-a1e7-ba29ff4eceb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:11.819384 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7b1a4-0c95-41fc-a1e7-ba29ff4eceb2 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:11.819732 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7b1a4-0c95-41fc-a1e7-ba29ff4eceb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:11.819963 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7b1a4-0c95-41fc-a1e7-ba29ff4eceb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:11.821016 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3592199-d1b2-42c2-90f2-6897bd59d720 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:11.830909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7b1a4-0c95-41fc-a1e7-ba29ff4eceb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:11.832469 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7b1a4-0c95-41fc-a1e7-ba29ff4eceb2 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:11.848812 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-75b7b1a4-0c95-41fc-a1e7-ba29ff4eceb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:11.848812 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:11.848812 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:11.848812 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:11.848812 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:11.848812 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:11.848812 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:11.848812 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:11.848812 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:11.848812 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:11.848812 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:11.848812 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:11.848812 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:11.848812 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:11.848812 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:11.848812 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:11.849844 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:11.849844 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:11.849844 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:11.849844 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:11.849844 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:11.849844 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:11.849844 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:11.849844 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:11.849844 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:11.849844 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:11.849844 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:11.849844 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:11.849844 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:11.849844 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:11.849844 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:11.849844 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:11.849844 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:11.850767 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:11.850767 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:11.850767 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:11.850767 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 323/651] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:11.850767 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7b1a4-0c95-41fc-a1e7-ba29ff4eceb2 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:11.864018 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea55e4-d284-4f76-b750-c320d9fb618e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:11.864926 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea55e4-d284-4f76-b750-c320d9fb618e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:11.864926 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea55e4-d284-4f76-b750-c320d9fb618e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:11.865180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea55e4-d284-4f76-b750-c320d9fb618e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:11.872645 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea55e4-d284-4f76-b750-c320d9fb618e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:11.874092 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea55e4-d284-4f76-b750-c320d9fb618e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:11.886793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-93ea55e4-d284-4f76-b750-c320d9fb618e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 12:06:11.886793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:11.886793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:11.886793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:11.886793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:11.886793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:11.886793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:11.886793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:11.886793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:11.886793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:11.886793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:11.886793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:11.886793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:06:11.886793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:06:11.886793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:06:11.886793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:06:11.888197 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:11.888197 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:11.888197 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:06:11.888197 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:11.888197 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:06:11.888197 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:11.888197 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:11.888197 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:11.888197 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:11.888197 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:11.888197 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:11.888197 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:11.888197 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:11.888197 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:11.888197 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:11.888197 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:11.888197 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:11.888197 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:11.889187 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:11.889187 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:11.889187 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:06:11.889187 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:06:11.889187 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:06:11.889187 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:06:11.889187 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:06:11.889187 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:06:11.889187 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 12:06:11.889187 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:11.893265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 329/652] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:06:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:11.894174 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea55e4-d284-4f76-b750-c320d9fb618e None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:11.959687 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7b1a4-0c95-41fc-a1e7-ba29ff4eceb2 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:11.971484 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:11.994603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c1cf21-dd4b-4ceb-a1bd-0a0be67b5e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:11.995228 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c1cf21-dd4b-4ceb-a1bd-0a0be67b5e9a None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:11.995503 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c1cf21-dd4b-4ceb-a1bd-0a0be67b5e9a None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:11.995800 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c1cf21-dd4b-4ceb-a1bd-0a0be67b5e9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:11.997596 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea55e4-d284-4f76-b750-c320d9fb618e None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:12.006744 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c1cf21-dd4b-4ceb-a1bd-0a0be67b5e9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:12.008358 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c1cf21-dd4b-4ceb-a1bd-0a0be67b5e9a None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:12.022597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-f2c1cf21-dd4b-4ceb-a1bd-0a0be67b5e9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:12.022597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:12.022597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:12.022597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:12.022597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:12.022597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:12.022597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:12.022597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:12.022597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:12.022597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:12.022597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:12.022597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:12.022597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:12.022597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:12.022597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:12.022597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:12.023755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:12.023755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:12.023755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:12.023755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:12.023755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:12.023755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:12.023755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:12.023755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:12.023755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:12.023755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:12.023755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:12.023755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:12.023755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:12.023755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:12.023755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:12.023755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:12.023755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:12.024660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:12.024660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:12.024660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:12.029637 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 324/653] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:12.030192 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c1cf21-dd4b-4ceb-a1bd-0a0be67b5e9a None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:12.032092 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4920a6d-d355-4675-b581-df85cd4501cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:12.032728 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4920a6d-d355-4675-b581-df85cd4501cc None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:12.033009 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4920a6d-d355-4675-b581-df85cd4501cc None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:12.033281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4920a6d-d355-4675-b581-df85cd4501cc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:12.041352 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4920a6d-d355-4675-b581-df85cd4501cc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:12.041665 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:12.042513 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4920a6d-d355-4675-b581-df85cd4501cc None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:12.054594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-c4920a6d-d355-4675-b581-df85cd4501cc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 05 12:06:12.054594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:12.054594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:12.054594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:12.054594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:12.054594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:12.054594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:12.054594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:12.054594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:12.054594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:12.054594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:12.054594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:12.054594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:06:12.054594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:06:12.054594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:06:12.054594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:06:12.055675 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:12.055675 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:12.055675 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:06:12.055675 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:12.055675 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:06:12.055675 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:12.055675 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:12.055675 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:12.055675 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:12.055675 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:12.055675 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:12.055675 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:12.055675 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:12.055675 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:12.055675 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:12.055675 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:12.055675 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:12.055675 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:12.056705 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:12.056705 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:12.056705 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:06:12.056705 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:06:12.056705 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:06:12.056705 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:06:12.056705 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:06:12.056705 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:06:12.056705 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 05 12:06:12.056705 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:12.059668 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 330/654] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Sep 5 12:06:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:12.060976 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4920a6d-d355-4675-b581-df85cd4501cc None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:12.139645 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c1cf21-dd4b-4ceb-a1bd-0a0be67b5e9a None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:12.176798 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f34680-8046-46bc-b052-68e5ff0a0dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:12.177407 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f34680-8046-46bc-b052-68e5ff0a0dce None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:12.177681 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f34680-8046-46bc-b052-68e5ff0a0dce None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:12.177949 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f34680-8046-46bc-b052-68e5ff0a0dce None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:12.178619 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f34680-8046-46bc-b052-68e5ff0a0dce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:12.178917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f34680-8046-46bc-b052-68e5ff0a0dce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:12.180305 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f34680-8046-46bc-b052-68e5ff0a0dce None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:12.181746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4920a6d-d355-4675-b581-df85cd4501cc None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:12.192950 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 325/655] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:12.194413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665ad14-0a3a-4936-9652-5ae2e86cc0a6 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:12.194703 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665ad14-0a3a-4936-9652-5ae2e86cc0a6 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:12.194977 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665ad14-0a3a-4936-9652-5ae2e86cc0a6 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:12.196435 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 326/656] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:12.200373 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665ad14-0a3a-4936-9652-5ae2e86cc0a6 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:12.217107 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9407b-f189-4a63-ad5a-80694cbc2046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:12.217643 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9407b-f189-4a63-ad5a-80694cbc2046 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:12.217924 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9407b-f189-4a63-ad5a-80694cbc2046 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:12.218206 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9407b-f189-4a63-ad5a-80694cbc2046 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:12.218930 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce9407b-f189-4a63-ad5a-80694cbc2046 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:12.219240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce9407b-f189-4a63-ad5a-80694cbc2046 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:12.220707 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce9407b-f189-4a63-ad5a-80694cbc2046 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:12.222367 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:12.239283 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 331/657] 10.222.0.16 () {56 vars in 1137 bytes} [Mon Sep 5 12:06:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:12.239961 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9407b-f189-4a63-ad5a-80694cbc2046 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:12.271684 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:12.305971 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665ad14-0a3a-4936-9652-5ae2e86cc0a6 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:12.340063 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f121be22-ffa3-48fa-864b-6e172dcbf40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:12.340667 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121be22-ffa3-48fa-864b-6e172dcbf40d None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:12.340944 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121be22-ffa3-48fa-864b-6e172dcbf40d None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:12.341790 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121be22-ffa3-48fa-864b-6e172dcbf40d None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:12.342441 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f121be22-ffa3-48fa-864b-6e172dcbf40d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:12.342744 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f121be22-ffa3-48fa-864b-6e172dcbf40d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:12.343812 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9407b-f189-4a63-ad5a-80694cbc2046 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:12.344153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f121be22-ffa3-48fa-864b-6e172dcbf40d None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:12.356449 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 327/658] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:12.357455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f121be22-ffa3-48fa-864b-6e172dcbf40d None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:12.384860 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c842b-80c9-49c1-ac57-60b0a08b0083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:12.385731 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c842b-80c9-49c1-ac57-60b0a08b0083 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:12.386016 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c842b-80c9-49c1-ac57-60b0a08b0083 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:12.386289 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c842b-80c9-49c1-ac57-60b0a08b0083 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:12.387014 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c842b-80c9-49c1-ac57-60b0a08b0083 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:12.387324 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c842b-80c9-49c1-ac57-60b0a08b0083 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:12.389331 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c842b-80c9-49c1-ac57-60b0a08b0083 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:12.407633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 332/659] 10.222.0.16 () {56 vars in 1139 bytes} [Mon Sep 5 12:06:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:12.408889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ae510-33c2-4250-aaf3-6ade0eda9292 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:12.409237 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ae510-33c2-4250-aaf3-6ade0eda9292 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:12.409518 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ae510-33c2-4250-aaf3-6ade0eda9292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:12.430299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-3d9ae510-33c2-4250-aaf3-6ade0eda9292 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:12.431515 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3d9ae510-33c2-4250-aaf3-6ade0eda9292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:12.461932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f121be22-ffa3-48fa-864b-6e172dcbf40d None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:12.477861 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:12.488421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:12.496604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f0134-10d6-4afc-aa4c-30b1ec7ceb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:12.497225 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f0134-10d6-4afc-aa4c-30b1ec7ceb76 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:12.497534 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f0134-10d6-4afc-aa4c-30b1ec7ceb76 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:12.497805 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f0134-10d6-4afc-aa4c-30b1ec7ceb76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:12.499982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 333/660] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:06:12.501147 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ae510-33c2-4250-aaf3-6ade0eda9292 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:12.504551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f0134-10d6-4afc-aa4c-30b1ec7ceb76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:12.505986 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f0134-10d6-4afc-aa4c-30b1ec7ceb76 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:12.517752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-7d6f0134-10d6-4afc-aa4c-30b1ec7ceb76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:12.517752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:12.517752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:12.517752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:12.517752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:12.517752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:12.517752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:12.517752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:12.517752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:12.517752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:12.517752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:12.517752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:12.517752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:06:12.517752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:06:12.517752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:06:12.518889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:06:12.518889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:12.518889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:12.518889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:12.518889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:12.518889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:12.518889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:12.518889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:12.518889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:12.518889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:12.518889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:12.518889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:12.518889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:12.518889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:12.518889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:12.518889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:12.518889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:12.519863 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:06:12.519863 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:06:12.519863 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:06:12.519863 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:06:12.519863 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:06:12.519863 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:06:12.519863 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:12.519863 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:12.519863 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 328/661] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:06:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:12.519863 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f0134-10d6-4afc-aa4c-30b1ec7ceb76 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:12.611977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ae510-33c2-4250-aaf3-6ade0eda9292 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:12.625415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f0134-10d6-4afc-aa4c-30b1ec7ceb76 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:12.650504 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97077c-e54c-488c-9eeb-9890728ed93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:12.651122 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97077c-e54c-488c-9eeb-9890728ed93d None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:12.651401 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97077c-e54c-488c-9eeb-9890728ed93d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:12.651709 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97077c-e54c-488c-9eeb-9890728ed93d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:12.661960 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e405238-c74c-4901-840f-1b06e91113a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:12.662686 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e405238-c74c-4901-840f-1b06e91113a3 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:12.663029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e405238-c74c-4901-840f-1b06e91113a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:12.663364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e405238-c74c-4901-840f-1b06e91113a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:12.670342 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e405238-c74c-4901-840f-1b06e91113a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:12.671918 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e405238-c74c-4901-840f-1b06e91113a3 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:12.683376 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-6e405238-c74c-4901-840f-1b06e91113a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:12.683376 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:12.683376 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:12.683376 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:12.683376 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:12.683376 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:12.683376 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:12.683376 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:12.683376 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:12.683376 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:12.683376 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:12.683376 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:12.683376 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:06:12.683376 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:06:12.683376 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:06:12.684497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:06:12.684497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:12.684497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:12.684497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:12.684497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:12.684497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:12.684497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:12.684497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:12.684497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:12.684497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:12.684497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:12.684497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:12.684497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:12.684497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:12.684497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:12.684497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:12.684497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:12.686237 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:06:12.686237 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:06:12.686237 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:06:12.686237 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:06:12.686237 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:06:12.686237 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:06:12.686237 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:12.686237 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:12.687361 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 329/662] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:06:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:12.692658 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e405238-c74c-4901-840f-1b06e91113a3 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:12.707582 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:12.729066 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c97077c-e54c-488c-9eeb-9890728ed93d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:12.730549 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c97077c-e54c-488c-9eeb-9890728ed93d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:12.745319 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:12.812705 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e405238-c74c-4901-840f-1b06e91113a3 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:12.837948 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 334/663] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:06:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:06:12.838885 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97077c-e54c-488c-9eeb-9890728ed93d None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:12.849672 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee76919-d60a-4bc6-bd24-11b294b6b44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:12.850382 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee76919-d60a-4bc6-bd24-11b294b6b44c None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:12.850658 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee76919-d60a-4bc6-bd24-11b294b6b44c None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:12.850932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee76919-d60a-4bc6-bd24-11b294b6b44c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:12.916426 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:12.920398 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee76919-d60a-4bc6-bd24-11b294b6b44c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:12.922077 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee76919-d60a-4bc6-bd24-11b294b6b44c None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:12.936180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97077c-e54c-488c-9eeb-9890728ed93d None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:12.970754 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce4658-3e88-4c9b-b7c7-a8eff4a54d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:12.970969 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:12.971617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce4658-3e88-4c9b-b7c7-a8eff4a54d84 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:12.971726 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce4658-3e88-4c9b-b7c7-a8eff4a54d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:12.971987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce4658-3e88-4c9b-b7c7-a8eff4a54d84 None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:12.972626 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fce4658-3e88-4c9b-b7c7-a8eff4a54d84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:12.972981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fce4658-3e88-4c9b-b7c7-a8eff4a54d84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:12.974387 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fce4658-3e88-4c9b-b7c7-a8eff4a54d84 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:12.986666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 335/664] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:06:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:12.987426 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce4658-3e88-4c9b-b7c7-a8eff4a54d84 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:13.023887 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 330/665] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:06:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:06:13.024745 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee76919-d60a-4bc6-bd24-11b294b6b44c None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:13.090897 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce4658-3e88-4c9b-b7c7-a8eff4a54d84 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:13.127507 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee76919-d60a-4bc6-bd24-11b294b6b44c None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:13.129829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d3bd86-6ee7-464f-bd05-86fcdf57e346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:13.130472 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3bd86-6ee7-464f-bd05-86fcdf57e346 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:13.130757 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3bd86-6ee7-464f-bd05-86fcdf57e346 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:13.131028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3bd86-6ee7-464f-bd05-86fcdf57e346 None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:13.131832 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d3bd86-6ee7-464f-bd05-86fcdf57e346 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:13.132150 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d3bd86-6ee7-464f-bd05-86fcdf57e346 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:13.133559 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d3bd86-6ee7-464f-bd05-86fcdf57e346 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:13.143005 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:13.146887 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 336/666] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:06:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:13.147921 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d3bd86-6ee7-464f-bd05-86fcdf57e346 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:13.176636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d5be52-c9d1-4ebf-8005-b1a51074ab99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:13.177263 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5be52-c9d1-4ebf-8005-b1a51074ab99 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:13.177537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5be52-c9d1-4ebf-8005-b1a51074ab99 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:13.177814 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5be52-c9d1-4ebf-8005-b1a51074ab99 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:13.188458 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d5be52-c9d1-4ebf-8005-b1a51074ab99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:06:13.193288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-d7d5be52-c9d1-4ebf-8005-b1a51074ab99 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:06:13.193288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:13.193288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:13.193288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:13.193288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:13.193288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:13.193288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:13.193288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:13.193288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:13.193288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:13.193288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:13.193288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:13.193288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:06:13.193288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:06:13.193288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:06:13.193288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:06:13.194410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:06:13.194410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:06:13.194410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:06:13.194410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:06:13.194410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:13.194410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:13.194410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:13.194410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:13.194410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:13.194410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:13.194410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:13.194410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:13.194410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:13.194410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:13.194410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:13.194410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:13.195278 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:13.195278 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:13.195278 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:13.195278 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:13.195278 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:06:13.195278 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:06:13.195278 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:06:13.195278 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:06:13.195278 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:06:13.195278 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:06:13.195278 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:06:13.195278 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:13.195278 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 331/667] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Sep 5 12:06:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:13.196134 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d5be52-c9d1-4ebf-8005-b1a51074ab99 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:13.208008 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:13.259553 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d3bd86-6ee7-464f-bd05-86fcdf57e346 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:13.294143 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51aafca-8d7e-4388-8254-d3413c97e898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:13.295303 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51aafca-8d7e-4388-8254-d3413c97e898 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:13.295611 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51aafca-8d7e-4388-8254-d3413c97e898 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:13.295886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51aafca-8d7e-4388-8254-d3413c97e898 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:13.298212 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d5be52-c9d1-4ebf-8005-b1a51074ab99 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:13.305924 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51aafca-8d7e-4388-8254-d3413c97e898 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:13.307360 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51aafca-8d7e-4388-8254-d3413c97e898 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:13.321744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-e51aafca-8d7e-4388-8254-d3413c97e898 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:13.321744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:13.321744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:13.321744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:13.321744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:13.321744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:13.321744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:13.321744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:13.321744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:13.321744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:13.321744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:13.321744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:13.321744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:13.321744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:13.321744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:13.321744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:13.322882 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:13.322882 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:13.322882 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:13.322882 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:13.322882 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:13.322882 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:13.322882 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:13.322882 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:13.322882 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:13.322882 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:13.322882 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:13.322882 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:13.322882 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:13.322882 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:13.322882 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:13.322882 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:13.322882 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:13.324006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:13.324006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:13.324006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:13.324006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 337/668] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:13.324006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51aafca-8d7e-4388-8254-d3413c97e898 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:13.334876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a24ccf-f824-4882-ae57-922965378dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:13.335480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a24ccf-f824-4882-ae57-922965378dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:13.335792 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a24ccf-f824-4882-ae57-922965378dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:13.336063 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a24ccf-f824-4882-ae57-922965378dd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:13.336792 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a24ccf-f824-4882-ae57-922965378dd7 None admin] RBAC: Adding query filter params (role.id=8cf764a0144a490a822fcf86d7aceeb0, scope.project.id=fa3178684d184ec880f17640d19d55b7, user.id=ed75c0be809d4a0eb0e2ccc59805b1ad) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:13.337099 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a24ccf-f824-4882-ae57-922965378dd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:13.338516 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a24ccf-f824-4882-ae57-922965378dd7 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:13.344710 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 332/669] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:06:13 2022] GET /identity/v3/role_assignments?user.id=ed75c0be809d4a0eb0e2ccc59805b1ad&scope.project.id=fa3178684d184ec880f17640d19d55b7&role.id=8cf764a0144a490a822fcf86d7aceeb0 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:13.345487 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a24ccf-f824-4882-ae57-922965378dd7 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:13.387894 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:13.439632 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:13.444880 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51aafca-8d7e-4388-8254-d3413c97e898 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:13.473927 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a24ccf-f824-4882-ae57-922965378dd7 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:13.481421 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8bbbbd-14d6-4323-96c6-f0da0ded17c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:13.482620 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bbbbd-14d6-4323-96c6-f0da0ded17c0 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:13.482900 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bbbbd-14d6-4323-96c6-f0da0ded17c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:13.483701 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bbbbd-14d6-4323-96c6-f0da0ded17c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:13.494125 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8bbbbd-14d6-4323-96c6-f0da0ded17c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:13.495728 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8bbbbd-14d6-4323-96c6-f0da0ded17c0 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:13.513578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-ec8bbbbd-14d6-4323-96c6-f0da0ded17c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:13.513578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:13.513578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:13.513578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:13.513578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:13.513578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:13.513578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:13.513578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:13.513578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:13.513578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:13.513578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:13.513578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:13.513578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:13.513578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:13.513578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:13.513578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:13.515098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:13.515098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:13.515098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:13.515098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:13.515098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:13.515098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:13.515098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:13.515098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:13.515098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:13.515098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:13.515098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:13.515098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:13.515098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:13.515098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:13.515098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:13.515098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:13.515098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:13.516242 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:13.516242 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:13.516242 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:13.516242 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 338/670] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:13.516242 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8bbbbd-14d6-4323-96c6-f0da0ded17c0 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:13.518626 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669df7b-417f-4dd5-8a9e-98567be6679e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:13.519422 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669df7b-417f-4dd5-8a9e-98567be6679e None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:13.519728 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669df7b-417f-4dd5-8a9e-98567be6679e None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:13.519997 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669df7b-417f-4dd5-8a9e-98567be6679e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:13.520721 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669df7b-417f-4dd5-8a9e-98567be6679e None admin] RBAC: Adding query filter params (role.id=8cf764a0144a490a822fcf86d7aceeb0, scope.project.id=fa3178684d184ec880f17640d19d55b7, user.id=7cda290418504f0db85d665479a05353) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:13.521029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669df7b-417f-4dd5-8a9e-98567be6679e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:13.522449 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669df7b-417f-4dd5-8a9e-98567be6679e None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:13.529523 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 333/671] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:06:13 2022] GET /identity/v3/role_assignments?user.id=7cda290418504f0db85d665479a05353&scope.project.id=fa3178684d184ec880f17640d19d55b7&role.id=8cf764a0144a490a822fcf86d7aceeb0 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:13.530354 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669df7b-417f-4dd5-8a9e-98567be6679e None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:13.658065 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8bbbbd-14d6-4323-96c6-f0da0ded17c0 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:13.661103 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:13.673357 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669df7b-417f-4dd5-8a9e-98567be6679e None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:13.710316 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb91a18-4f79-48f0-9272-f4a2b2665974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:13.710954 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb91a18-4f79-48f0-9272-f4a2b2665974 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:13.711234 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb91a18-4f79-48f0-9272-f4a2b2665974 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:13.711511 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb91a18-4f79-48f0-9272-f4a2b2665974 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:13.712252 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb91a18-4f79-48f0-9272-f4a2b2665974 None admin] RBAC: Adding query filter params (name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:13.712559 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb91a18-4f79-48f0-9272-f4a2b2665974 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:13.713995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb91a18-4f79-48f0-9272-f4a2b2665974 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:13.726060 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 339/672] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Sep 5 12:06:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:13.726662 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b431f970-71d1-4b32-8669-ff86d4ab8de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:13.726950 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb91a18-4f79-48f0-9272-f4a2b2665974 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:13.727844 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b431f970-71d1-4b32-8669-ff86d4ab8de9 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:13.728187 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b431f970-71d1-4b32-8669-ff86d4ab8de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:13.728612 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b431f970-71d1-4b32-8669-ff86d4ab8de9 None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:13.729385 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b431f970-71d1-4b32-8669-ff86d4ab8de9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:13.729841 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b431f970-71d1-4b32-8669-ff86d4ab8de9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:13.731250 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b431f970-71d1-4b32-8669-ff86d4ab8de9 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:13.734408 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:13.749277 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 334/673] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:13.750875 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b431f970-71d1-4b32-8669-ff86d4ab8de9 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:13.859061 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb91a18-4f79-48f0-9272-f4a2b2665974 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:13.882234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b431f970-71d1-4b32-8669-ff86d4ab8de9 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:13.897284 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c6423-06f1-4e2b-ad48-4104a39d9e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:13.897945 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c6423-06f1-4e2b-ad48-4104a39d9e4c None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:13.898289 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c6423-06f1-4e2b-ad48-4104a39d9e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:13.898626 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c6423-06f1-4e2b-ad48-4104a39d9e4c None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:13.899606 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c6423-06f1-4e2b-ad48-4104a39d9e4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:13.899971 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c6423-06f1-4e2b-ad48-4104a39d9e4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:13.901418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c6423-06f1-4e2b-ad48-4104a39d9e4c None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:13.916327 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 340/674] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:13.917381 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c6423-06f1-4e2b-ad48-4104a39d9e4c None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:13.928454 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f3e80-169c-4134-aeef-ba8ae2cd7a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:13.929383 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f3e80-169c-4134-aeef-ba8ae2cd7a89 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:13.929864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f3e80-169c-4134-aeef-ba8ae2cd7a89 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:13.930308 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f3e80-169c-4134-aeef-ba8ae2cd7a89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:13.942011 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f3e80-169c-4134-aeef-ba8ae2cd7a89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:13.943882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f3e80-169c-4134-aeef-ba8ae2cd7a89 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:13.952612 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:13.962813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-bd2f3e80-169c-4134-aeef-ba8ae2cd7a89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:13.962813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:13.962813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:13.962813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:13.962813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:13.962813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:13.962813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:13.962813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:13.962813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:13.962813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:13.962813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:13.962813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:13.962813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:13.962813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:13.962813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:13.962813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:13.963882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:13.963882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:13.963882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:13.963882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:13.963882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:13.963882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:13.963882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:13.963882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:13.963882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:13.963882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:13.963882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:13.963882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:13.963882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:13.963882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:13.963882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:13.963882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:13.963882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:13.965013 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:13.965013 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:13.965013 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:13.967370 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 335/675] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:13.968371 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f3e80-169c-4134-aeef-ba8ae2cd7a89 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:14.022341 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:14.035639 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c6423-06f1-4e2b-ad48-4104a39d9e4c None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:14.073196 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5875fd91-fc54-47c1-92b6-c8d9dd445ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:14.074028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875fd91-fc54-47c1-92b6-c8d9dd445ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:14.074371 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875fd91-fc54-47c1-92b6-c8d9dd445ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:14.074720 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875fd91-fc54-47c1-92b6-c8d9dd445ae3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:14.082031 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f3e80-169c-4134-aeef-ba8ae2cd7a89 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:14.082687 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5875fd91-fc54-47c1-92b6-c8d9dd445ae3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:14.084411 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5875fd91-fc54-47c1-92b6-c8d9dd445ae3 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:14.096548 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-5875fd91-fc54-47c1-92b6-c8d9dd445ae3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:14.096548 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:14.096548 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:14.096548 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:14.096548 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:14.096548 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:14.096548 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:14.096548 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:14.096548 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:14.096548 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:14.096548 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:14.096548 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:14.096548 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:06:14.096548 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:06:14.096548 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:06:14.097518 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:06:14.097518 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:14.097518 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:14.097518 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:14.097518 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:14.097518 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:14.097518 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:14.097518 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:14.097518 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:14.097518 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:14.097518 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:14.097518 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:14.097518 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:14.097518 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:14.097518 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:14.097518 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:14.097518 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:14.098454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:06:14.098454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:06:14.098454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:06:14.098454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:06:14.098454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:06:14.098454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:06:14.098454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:14.098454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:14.098454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 341/676] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:06:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:14.098951 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5875fd91-fc54-47c1-92b6-c8d9dd445ae3 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:14.131115 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fe987-c610-452e-aed5-9c971812c12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:14.132172 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fe987-c610-452e-aed5-9c971812c12e None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:14.132760 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fe987-c610-452e-aed5-9c971812c12e None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:14.133191 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fe987-c610-452e-aed5-9c971812c12e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:14.143903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59fe987-c610-452e-aed5-9c971812c12e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:14.146224 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59fe987-c610-452e-aed5-9c971812c12e None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:14.162909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-b59fe987-c610-452e-aed5-9c971812c12e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:14.162909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:14.162909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:14.162909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:14.162909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:14.162909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:14.162909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:14.162909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:14.162909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:14.162909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:14.162909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:14.162909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:14.162909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:06:14.162909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:06:14.162909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:06:14.164299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:06:14.164299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:14.164299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:14.164299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:14.164299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:14.164299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:14.164299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:14.164299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:14.164299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:14.164299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:14.164299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:14.164299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:14.164299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:14.164299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:14.164299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:14.164299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:14.164299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:14.165560 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:06:14.165560 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:06:14.165560 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:06:14.165560 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:06:14.165560 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:06:14.165560 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:06:14.165560 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:14.165560 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:14.168518 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 336/677] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:06:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:14.169935 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fe987-c610-452e-aed5-9c971812c12e None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:14.207929 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:14.220862 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5875fd91-fc54-47c1-92b6-c8d9dd445ae3 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:14.259942 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072fb8d9-bc11-47e9-9ddc-9f95a4a1b8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:14.260700 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072fb8d9-bc11-47e9-9ddc-9f95a4a1b8c3 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:14.260988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072fb8d9-bc11-47e9-9ddc-9f95a4a1b8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:14.261265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072fb8d9-bc11-47e9-9ddc-9f95a4a1b8c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:14.262062 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072fb8d9-bc11-47e9-9ddc-9f95a4a1b8c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:14.262308 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072fb8d9-bc11-47e9-9ddc-9f95a4a1b8c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:14.264292 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072fb8d9-bc11-47e9-9ddc-9f95a4a1b8c3 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:14.278539 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fe987-c610-452e-aed5-9c971812c12e None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:14.282416 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 342/678] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:14.283323 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072fb8d9-bc11-47e9-9ddc-9f95a4a1b8c3 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:14.288813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:14.315255 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caaa502-fe7f-418d-aa8e-5d96f00231d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:14.316110 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caaa502-fe7f-418d-aa8e-5d96f00231d4 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:14.316479 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caaa502-fe7f-418d-aa8e-5d96f00231d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:14.316852 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caaa502-fe7f-418d-aa8e-5d96f00231d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:14.317634 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caaa502-fe7f-418d-aa8e-5d96f00231d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:14.318126 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caaa502-fe7f-418d-aa8e-5d96f00231d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:14.319667 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caaa502-fe7f-418d-aa8e-5d96f00231d4 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:14.332966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 337/679] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:06:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:14.334144 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caaa502-fe7f-418d-aa8e-5d96f00231d4 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:14.415018 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072fb8d9-bc11-47e9-9ddc-9f95a4a1b8c3 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:14.446685 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caaa502-fe7f-418d-aa8e-5d96f00231d4 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:14.457937 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f95fc0-e4c6-412f-b71a-06cef67231eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:14.458828 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f95fc0-e4c6-412f-b71a-06cef67231eb None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:14.459331 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f95fc0-e4c6-412f-b71a-06cef67231eb None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:14.459870 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f95fc0-e4c6-412f-b71a-06cef67231eb None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:14.460751 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f95fc0-e4c6-412f-b71a-06cef67231eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:14.461243 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f95fc0-e4c6-412f-b71a-06cef67231eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:14.463061 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f95fc0-e4c6-412f-b71a-06cef67231eb None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:14.478292 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 343/680] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:06:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:14.479546 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f95fc0-e4c6-412f-b71a-06cef67231eb None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:14.487768 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2801bf-adea-45db-9ed2-b0e0a53c424c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:14.488556 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2801bf-adea-45db-9ed2-b0e0a53c424c None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:14.488926 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2801bf-adea-45db-9ed2-b0e0a53c424c None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:14.489319 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2801bf-adea-45db-9ed2-b0e0a53c424c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:14.491453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:14.498482 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2801bf-adea-45db-9ed2-b0e0a53c424c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:14.500148 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2801bf-adea-45db-9ed2-b0e0a53c424c None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:14.513631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-be2801bf-adea-45db-9ed2-b0e0a53c424c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 05 12:06:14.513631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:14.513631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:14.513631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:14.513631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:14.513631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:14.513631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:14.513631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:14.513631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:14.513631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:14.513631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:14.513631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:14.513631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:06:14.513631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:06:14.513631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:06:14.513631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:06:14.515007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:14.515007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:14.515007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:06:14.515007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:14.515007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:06:14.515007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:14.515007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:14.515007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:14.515007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:14.515007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:14.515007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:14.515007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:14.515007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:14.515007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:14.515007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:14.515007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:14.515007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:14.515007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:14.516334 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:14.516334 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:14.516334 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:06:14.516334 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:06:14.516334 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:06:14.516334 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:06:14.516334 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:06:14.516334 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:06:14.516334 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 05 12:06:14.516334 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:14.521829 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 338/681] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:06:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:14.523037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2801bf-adea-45db-9ed2-b0e0a53c424c None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:14.555755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:14.608846 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f95fc0-e4c6-412f-b71a-06cef67231eb None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:14.640646 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2801bf-adea-45db-9ed2-b0e0a53c424c None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:14.653099 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16e6b9-ec5f-4496-830f-f74b4a504d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:14.654830 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16e6b9-ec5f-4496-830f-f74b4a504d9f None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:14.655368 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16e6b9-ec5f-4496-830f-f74b4a504d9f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:14.655844 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16e6b9-ec5f-4496-830f-f74b4a504d9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:14.656782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c16e6b9-ec5f-4496-830f-f74b4a504d9f None admin] RBAC: Adding query filter params (role.id=8cf764a0144a490a822fcf86d7aceeb0, scope.project.id=fa3178684d184ec880f17640d19d55b7, user.id=e8a47a0d9bfd473cbf0147d7801442d0) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:14.657276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c16e6b9-ec5f-4496-830f-f74b4a504d9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:14.659117 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c16e6b9-ec5f-4496-830f-f74b4a504d9f None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:14.666429 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 344/682] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:06:14 2022] GET /identity/v3/role_assignments?user.id=e8a47a0d9bfd473cbf0147d7801442d0&scope.project.id=fa3178684d184ec880f17640d19d55b7&role.id=8cf764a0144a490a822fcf86d7aceeb0 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:14.667516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16e6b9-ec5f-4496-830f-f74b4a504d9f None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:14.677098 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e65f6-467c-4fab-8e14-70900d1e0439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:14.677715 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e65f6-467c-4fab-8e14-70900d1e0439 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:14.678177 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e65f6-467c-4fab-8e14-70900d1e0439 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:14.678721 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e65f6-467c-4fab-8e14-70900d1e0439 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:14.679933 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e65f6-467c-4fab-8e14-70900d1e0439 None admin] RBAC: Adding query filter params (role.id=8cf764a0144a490a822fcf86d7aceeb0, scope.project.id=fa3178684d184ec880f17640d19d55b7, user.id=9f79599ed0b74bdbab198627ef72252a) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:14.679933 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e65f6-467c-4fab-8e14-70900d1e0439 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:14.681368 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e65f6-467c-4fab-8e14-70900d1e0439 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:14.688129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 339/683] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:06:14 2022] GET /identity/v3/role_assignments?user.id=9f79599ed0b74bdbab198627ef72252a&scope.project.id=fa3178684d184ec880f17640d19d55b7&role.id=8cf764a0144a490a822fcf86d7aceeb0 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:14.757494 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:14.798721 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16e6b9-ec5f-4496-830f-f74b4a504d9f None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:14.847398 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad8f06-ba30-4118-9823-c0de35bfebee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:14.848205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad8f06-ba30-4118-9823-c0de35bfebee None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:14.848580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad8f06-ba30-4118-9823-c0de35bfebee None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:14.848948 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad8f06-ba30-4118-9823-c0de35bfebee None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:14.849800 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad8f06-ba30-4118-9823-c0de35bfebee None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:14.850210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad8f06-ba30-4118-9823-c0de35bfebee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:14.851796 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad8f06-ba30-4118-9823-c0de35bfebee None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:14.877181 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 345/684] 10.222.0.16 () {56 vars in 1135 bytes} [Mon Sep 5 12:06:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:14.883103 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e65f6-467c-4fab-8e14-70900d1e0439 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:15.002653 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:15.014537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e65f6-467c-4fab-8e14-70900d1e0439 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:15.053123 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12340ea1-dff6-4c13-bce9-f3bf5ff83ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:15.053757 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12340ea1-dff6-4c13-bce9-f3bf5ff83ffc None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:15.054101 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12340ea1-dff6-4c13-bce9-f3bf5ff83ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:15.054376 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12340ea1-dff6-4c13-bce9-f3bf5ff83ffc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:15.141746 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12340ea1-dff6-4c13-bce9-f3bf5ff83ffc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:15.143217 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12340ea1-dff6-4c13-bce9-f3bf5ff83ffc None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:15.264702 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:15.276311 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 340/685] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:06:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:06:15.281967 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad8f06-ba30-4118-9823-c0de35bfebee None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:15.396400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad8f06-ba30-4118-9823-c0de35bfebee None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:15.425385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:15.439068 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2921a46c-1ad4-499e-bd67-b432f1ac3053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:15.439735 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921a46c-1ad4-499e-bd67-b432f1ac3053 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:15.440014 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921a46c-1ad4-499e-bd67-b432f1ac3053 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:15.440286 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921a46c-1ad4-499e-bd67-b432f1ac3053 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:15.451223 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2921a46c-1ad4-499e-bd67-b432f1ac3053 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:15.452711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2921a46c-1ad4-499e-bd67-b432f1ac3053 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:15.476033 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-2921a46c-1ad4-499e-bd67-b432f1ac3053 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:15.476033 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:15.476033 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:15.476033 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:15.476033 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:15.476033 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:15.476033 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:15.476033 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:15.476033 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:15.476033 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:15.476033 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:15.476033 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:15.476033 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:15.476033 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:15.476033 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:15.476033 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:15.477075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:15.477075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:15.477075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:15.477075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:15.477075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:15.477075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:15.477075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:15.477075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:15.477075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:15.477075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:15.477075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:15.477075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:15.477075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:15.477075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:15.477075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:15.477075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:15.477075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:15.478788 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:15.478788 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:15.478788 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:15.478788 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 346/686] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:15.484644 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12340ea1-dff6-4c13-bce9-f3bf5ff83ffc None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:15.574534 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e0a7a-7abb-4989-8e38-280e09cb13c6 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:15.574880 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e0a7a-7abb-4989-8e38-280e09cb13c6 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:15.575087 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e0a7a-7abb-4989-8e38-280e09cb13c6 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:15.576655 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 347/687] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:06:15.582291 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fddea-7b40-4519-aeb7-0f36dcf2f119 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:15.582875 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fddea-7b40-4519-aeb7-0f36dcf2f119 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:15.583380 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fddea-7b40-4519-aeb7-0f36dcf2f119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:15.613205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-131fddea-7b40-4519-aeb7-0f36dcf2f119 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:15.616935 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-131fddea-7b40-4519-aeb7-0f36dcf2f119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:15.654155 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12340ea1-dff6-4c13-bce9-f3bf5ff83ffc None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:15.719677 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07f76c-e843-4f83-bec9-29e0f04c5818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:15.720344 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07f76c-e843-4f83-bec9-29e0f04c5818 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:15.720572 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07f76c-e843-4f83-bec9-29e0f04c5818 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:15.720814 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07f76c-e843-4f83-bec9-29e0f04c5818 None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:15.721464 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c07f76c-e843-4f83-bec9-29e0f04c5818 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:15.721763 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c07f76c-e843-4f83-bec9-29e0f04c5818 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:15.723107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c07f76c-e843-4f83-bec9-29e0f04c5818 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:15.729909 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 348/688] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:06:15.741388 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 341/689] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:15.748593 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131fddea-7b40-4519-aeb7-0f36dcf2f119 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:15.858778 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131fddea-7b40-4519-aeb7-0f36dcf2f119 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:15.870380 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:15.902919 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4232e1-d22b-469a-a348-a8e6cfbb7839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:15.904607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4232e1-d22b-469a-a348-a8e6cfbb7839 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:15.905093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4232e1-d22b-469a-a348-a8e6cfbb7839 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:15.905562 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4232e1-d22b-469a-a348-a8e6cfbb7839 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:15.918216 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4232e1-d22b-469a-a348-a8e6cfbb7839 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:15.920051 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4232e1-d22b-469a-a348-a8e6cfbb7839 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:15.934852 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-dc4232e1-d22b-469a-a348-a8e6cfbb7839 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:15.934852 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:15.934852 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:15.934852 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:15.934852 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:15.934852 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:15.934852 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:15.934852 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:15.934852 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:15.934852 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:15.934852 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:15.934852 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:15.934852 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:06:15.934852 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:06:15.934852 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:06:15.935889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:06:15.935889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:15.935889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:15.935889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:15.935889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:15.935889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:15.935889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:15.935889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:15.935889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:15.935889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:15.935889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:15.935889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:15.935889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:15.935889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:15.935889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:15.935889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:15.935889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:15.936825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:06:15.936825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:06:15.936825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:06:15.936825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:06:15.936825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:06:15.936825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:06:15.936825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:15.936825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:15.939280 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 349/690] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:06:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:15.947239 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07f76c-e843-4f83-bec9-29e0f04c5818 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:15.978889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1d225-001c-4218-ab99-ecbc2ea220b1 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:15.979564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1d225-001c-4218-ab99-ecbc2ea220b1 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:15.980068 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1d225-001c-4218-ab99-ecbc2ea220b1 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:15.981749 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 350/691] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:06:15.990941 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25717e-a365-4e37-81bb-c455d67d1f47 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:15.991598 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25717e-a365-4e37-81bb-c455d67d1f47 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:15.992086 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25717e-a365-4e37-81bb-c455d67d1f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:16.025882 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-6a25717e-a365-4e37-81bb-c455d67d1f47 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:16.028148 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:16.028367 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6a25717e-a365-4e37-81bb-c455d67d1f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:16.112924 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07f76c-e843-4f83-bec9-29e0f04c5818 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:16.164970 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 351/692] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:06:16.166060 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25717e-a365-4e37-81bb-c455d67d1f47 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:16.185242 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74994d4-05e1-43e9-af5a-848a6be61355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:16.186237 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74994d4-05e1-43e9-af5a-848a6be61355 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:16.186723 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74994d4-05e1-43e9-af5a-848a6be61355 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:16.187206 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74994d4-05e1-43e9-af5a-848a6be61355 None admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:16.188304 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74994d4-05e1-43e9-af5a-848a6be61355 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:16.188848 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74994d4-05e1-43e9-af5a-848a6be61355 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:16.190572 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74994d4-05e1-43e9-af5a-848a6be61355 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:16.220742 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 342/693] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:06:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:06:16.229997 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74994d4-05e1-43e9-af5a-848a6be61355 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:16.360501 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:16.365567 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25717e-a365-4e37-81bb-c455d67d1f47 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:16.404317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74994d4-05e1-43e9-af5a-848a6be61355 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:16.432943 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d2ddd-3cf6-4302-8b6f-d85204d324ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:16.433570 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d2ddd-3cf6-4302-8b6f-d85204d324ba None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:16.433845 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d2ddd-3cf6-4302-8b6f-d85204d324ba None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:16.434110 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d2ddd-3cf6-4302-8b6f-d85204d324ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:16.435855 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:16.452150 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d2ddd-3cf6-4302-8b6f-d85204d324ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:16.454408 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d2ddd-3cf6-4302-8b6f-d85204d324ba None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:16.463768 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c7e2a3-ad99-4aac-8e3d-c086a422a7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:16.464380 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7e2a3-ad99-4aac-8e3d-c086a422a7cc None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:16.464704 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7e2a3-ad99-4aac-8e3d-c086a422a7cc None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:16.464978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7e2a3-ad99-4aac-8e3d-c086a422a7cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:16.465708 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c7e2a3-ad99-4aac-8e3d-c086a422a7cc None admin] RBAC: Adding query filter params (role.id=8cf764a0144a490a822fcf86d7aceeb0, scope.project.id=fa3178684d184ec880f17640d19d55b7, user.id=857eb7657e7d45938baca17f91a16901) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:16.466546 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c7e2a3-ad99-4aac-8e3d-c086a422a7cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:16.468796 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c7e2a3-ad99-4aac-8e3d-c086a422a7cc None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:16.474748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-8b1d2ddd-3cf6-4302-8b6f-d85204d324ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:16.474748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:16.474748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:16.474748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:16.474748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:16.474748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:16.474748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:16.474748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:16.474748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:16.474748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:16.474748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:16.474748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:16.474748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:16.474748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:16.474748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:16.474748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:16.476006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:16.476006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:16.476006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:16.476006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:16.476006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:16.476006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:16.476006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:16.476006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:16.476006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:16.476006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:16.476006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:16.476006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:16.476006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:16.476006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:16.476006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:16.476006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:16.476006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:16.476919 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:16.476919 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:16.476919 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:16.477469 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 343/694] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:06:16 2022] GET /identity/v3/role_assignments?user.id=857eb7657e7d45938baca17f91a16901&scope.project.id=fa3178684d184ec880f17640d19d55b7&role.id=8cf764a0144a490a822fcf86d7aceeb0 => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:16.477469 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c7e2a3-ad99-4aac-8e3d-c086a422a7cc None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:16.479169 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 352/695] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:16.485390 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d2ddd-3cf6-4302-8b6f-d85204d324ba None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:16.677099 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d2ddd-3cf6-4302-8b6f-d85204d324ba None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:16.677715 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c7e2a3-ad99-4aac-8e3d-c086a422a7cc None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:16.720545 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e8bfe-18f9-48c6-aad2-0e06ae1f6b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:16.721254 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e8bfe-18f9-48c6-aad2-0e06ae1f6b56 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:16.721531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e8bfe-18f9-48c6-aad2-0e06ae1f6b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:16.721802 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e8bfe-18f9-48c6-aad2-0e06ae1f6b56 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:16.722552 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e8bfe-18f9-48c6-aad2-0e06ae1f6b56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:16.722668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb590e-6e8a-4d64-a935-1310d83a08b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:16.723082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e8bfe-18f9-48c6-aad2-0e06ae1f6b56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:16.723266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb590e-6e8a-4d64-a935-1310d83a08b2 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:16.723558 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb590e-6e8a-4d64-a935-1310d83a08b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:16.723861 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb590e-6e8a-4d64-a935-1310d83a08b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:16.724241 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e8bfe-18f9-48c6-aad2-0e06ae1f6b56 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:16.735028 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb590e-6e8a-4d64-a935-1310d83a08b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:06:16.738628 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 353/696] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:06:16.740485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-b6cb590e-6e8a-4d64-a935-1310d83a08b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:06:16.740485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:16.740485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:16.740485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:16.740485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:16.740485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:16.740485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:16.740485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:16.740485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:16.740485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:16.740485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:16.740485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:16.740485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:06:16.740485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:06:16.740485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:06:16.740485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:06:16.741428 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:06:16.741428 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:06:16.741428 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:06:16.741428 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:06:16.741428 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:16.741428 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:16.741428 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:16.741428 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:16.741428 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:16.741428 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:16.741428 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:16.741428 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:16.741428 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:16.741428 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:16.741428 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:16.741428 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:16.742829 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:16.742829 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:16.742829 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:16.742829 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:16.742829 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:06:16.742829 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:06:16.742829 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:06:16.742829 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:06:16.742829 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:06:16.742829 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:06:16.742829 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:06:16.742829 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:16.742829 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 344/697] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:06:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:16.747447 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb590e-6e8a-4d64-a935-1310d83a08b2 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:16.747590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e8bfe-18f9-48c6-aad2-0e06ae1f6b56 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:16.764872 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:16.793545 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:16.873606 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb590e-6e8a-4d64-a935-1310d83a08b2 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:16.875556 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e8bfe-18f9-48c6-aad2-0e06ae1f6b56 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:16.913829 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa8be86-d73f-48c1-bccb-f0e2c0170492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:16.914442 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8be86-d73f-48c1-bccb-f0e2c0170492 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:16.914720 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8be86-d73f-48c1-bccb-f0e2c0170492 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:16.915047 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8be86-d73f-48c1-bccb-f0e2c0170492 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:16.915755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa8be86-d73f-48c1-bccb-f0e2c0170492 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:16.916057 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa8be86-d73f-48c1-bccb-f0e2c0170492 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:16.916957 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7064106-9b23-4dc8-8df2-ef6b749f0b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:16.917496 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa8be86-d73f-48c1-bccb-f0e2c0170492 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:16.917617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7064106-9b23-4dc8-8df2-ef6b749f0b88 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:16.917849 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7064106-9b23-4dc8-8df2-ef6b749f0b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:16.918118 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7064106-9b23-4dc8-8df2-ef6b749f0b88 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:16.918856 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7064106-9b23-4dc8-8df2-ef6b749f0b88 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:16.921232 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7064106-9b23-4dc8-8df2-ef6b749f0b88 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:16.928411 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 345/698] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:06:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:16.934803 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa8be86-d73f-48c1-bccb-f0e2c0170492 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:16.952487 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 354/699] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Sep 5 12:06:16 2022] POST /identity/v3/users => generated 254 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:06:17.064044 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:17.068597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa8be86-d73f-48c1-bccb-f0e2c0170492 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:17.113171 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5b947-51c1-4916-998a-a1624e5f003d None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:17.113560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5b947-51c1-4916-998a-a1624e5f003d None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:17.113837 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5b947-51c1-4916-998a-a1624e5f003d None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:17.114550 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8326bd9-7772-4e47-ab90-88ae68b3b3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:17.115186 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8326bd9-7772-4e47-ab90-88ae68b3b3f8 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:17.115448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 355/700] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:17.115576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8326bd9-7772-4e47-ab90-88ae68b3b3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:17.115741 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8326bd9-7772-4e47-ab90-88ae68b3b3f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:17.128495 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8326bd9-7772-4e47-ab90-88ae68b3b3f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:17.128966 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee950a1-9ff7-46cc-94fb-0f9d0239bf97 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:17.129252 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee950a1-9ff7-46cc-94fb-0f9d0239bf97 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:17.129527 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee950a1-9ff7-46cc-94fb-0f9d0239bf97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:17.129940 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8326bd9-7772-4e47-ab90-88ae68b3b3f8 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:17.149266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-e8326bd9-7772-4e47-ab90-88ae68b3b3f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:17.149266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:17.149266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:17.149266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:17.149266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:17.149266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:17.149266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:17.149266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:17.149266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:17.149266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:17.149266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:17.149266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:17.149266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:17.149266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:17.149266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:17.149266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:17.150343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:17.150343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:17.150343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:17.150343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:17.150343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:17.150343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:17.150343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:17.150343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:17.150343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:17.150343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:17.150343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:17.150343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:17.150343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:17.150343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:17.150343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:17.150343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:17.150343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:17.151275 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:17.151275 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:17.151275 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:17.153928 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 346/701] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:17.159190 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8326bd9-7772-4e47-ab90-88ae68b3b3f8 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:17.163091 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-7ee950a1-9ff7-46cc-94fb-0f9d0239bf97 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:17.165646 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7ee950a1-9ff7-46cc-94fb-0f9d0239bf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:17.246990 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:17.254815 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 356/702] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:06:17.265526 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fee0fa-4c34-47e9-9bdc-0e674d5f35a1 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:17.265829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fee0fa-4c34-47e9-9bdc-0e674d5f35a1 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:17.266075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fee0fa-4c34-47e9-9bdc-0e674d5f35a1 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:17.267772 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 357/703] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:17.273873 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce24c16-ee9a-4a76-a5d9-bfa61f028826 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:17.274185 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce24c16-ee9a-4a76-a5d9-bfa61f028826 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:17.274419 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce24c16-ee9a-4a76-a5d9-bfa61f028826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:17.292704 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-6ce24c16-ee9a-4a76-a5d9-bfa61f028826 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:17.293891 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6ce24c16-ee9a-4a76-a5d9-bfa61f028826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:17.311713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8326bd9-7772-4e47-ab90-88ae68b3b3f8 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:17.362300 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6551804-224c-4ef5-af13-3fd5989a9549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:17.363757 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6551804-224c-4ef5-af13-3fd5989a9549 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:17.364488 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6551804-224c-4ef5-af13-3fd5989a9549 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:17.364947 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6551804-224c-4ef5-af13-3fd5989a9549 None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:17.366075 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6551804-224c-4ef5-af13-3fd5989a9549 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:17.367030 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6551804-224c-4ef5-af13-3fd5989a9549 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:17.369180 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6551804-224c-4ef5-af13-3fd5989a9549 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:17.377349 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 358/704] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:06:17.385095 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 347/705] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:17.390813 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce24c16-ee9a-4a76-a5d9-bfa61f028826 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:17.499093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce24c16-ee9a-4a76-a5d9-bfa61f028826 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:17.538472 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:17.548762 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d60d8-029a-4308-875f-3218da14d436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:17.549404 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d60d8-029a-4308-875f-3218da14d436 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:17.549685 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d60d8-029a-4308-875f-3218da14d436 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:17.549949 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d60d8-029a-4308-875f-3218da14d436 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:17.564331 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d60d8-029a-4308-875f-3218da14d436 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:17.565848 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d60d8-029a-4308-875f-3218da14d436 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:17.572457 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6551804-224c-4ef5-af13-3fd5989a9549 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:17.578483 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-035d60d8-029a-4308-875f-3218da14d436 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:06:17.578483 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:17.578483 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:17.578483 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:17.578483 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:17.578483 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:17.578483 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:17.578483 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:17.578483 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:17.578483 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:17.578483 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:17.578483 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:17.578483 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:06:17.578483 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:06:17.578483 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:06:17.578483 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:06:17.579463 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:17.579463 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:17.579463 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:06:17.579463 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:17.579463 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:06:17.579463 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:17.579463 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:17.579463 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:17.579463 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:17.579463 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:17.579463 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:17.579463 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:17.579463 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:17.579463 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:17.579463 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:17.579463 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:17.579463 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:17.579463 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:17.580765 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:17.580765 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:17.580765 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:06:17.580765 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:06:17.580765 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:06:17.580765 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:06:17.580765 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:06:17.580765 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:06:17.580765 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:06:17.580765 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:17.580765 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 359/706] 10.222.0.16 () {56 vars in 1093 bytes} [Mon Sep 5 12:06:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:17.585267 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d60d8-029a-4308-875f-3218da14d436 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:17.596710 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:17.731472 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6551804-224c-4ef5-af13-3fd5989a9549 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:17.766752 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d60d8-029a-4308-875f-3218da14d436 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:17.784853 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602637b4-ebdc-4c9f-9aec-f1f3b4fede3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:17.786019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602637b4-ebdc-4c9f-9aec-f1f3b4fede3d None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:17.786532 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602637b4-ebdc-4c9f-9aec-f1f3b4fede3d None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:17.787003 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602637b4-ebdc-4c9f-9aec-f1f3b4fede3d None admin] PATH_INFO: `/v3/services/network` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:17.787930 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602637b4-ebdc-4c9f-9aec-f1f3b4fede3d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:17.789629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602637b4-ebdc-4c9f-9aec-f1f3b4fede3d None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:17.803029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-602637b4-ebdc-4c9f-9aec-f1f3b4fede3d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 12:06:17.803029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:17.803029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:06:17.803029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:06:17.803029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:06:17.803029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:06:17.803029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:06:17.803029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:17.803029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 12:06:17.803029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:17.803029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:06:17.803029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:17.803029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:17.803029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:17.803029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:17.803029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:17.803029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:17.804216 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:17.804216 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:17.804216 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:17.804216 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:17.804216 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:17.804216 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:17.804216 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:06:17.804216 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:06:17.804216 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:06:17.804216 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:06:17.804216 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:17.804216 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:17.804216 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:17.804216 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:17.804216 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:17.804216 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:17.805071 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:17.805071 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:17.805071 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:17.805071 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:17.805071 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:17.805071 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:17.805071 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:17.805071 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:17.805071 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:17.805071 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:17.805071 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:06:17.805071 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:17.805071 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 12:06:17.805071 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:17.812748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd3155-c07a-4b69-aca4-e44e2ea86bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:17.812748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd3155-c07a-4b69-aca4-e44e2ea86bdd None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:17.812748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd3155-c07a-4b69-aca4-e44e2ea86bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:17.812748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd3155-c07a-4b69-aca4-e44e2ea86bdd None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:17.813157 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd3155-c07a-4b69-aca4-e44e2ea86bdd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:17.813157 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd3155-c07a-4b69-aca4-e44e2ea86bdd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:17.813814 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd3155-c07a-4b69-aca4-e44e2ea86bdd None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:17.815784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 348/707] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:06:17 2022] GET /identity/v3/services/network => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:17.816832 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602637b4-ebdc-4c9f-9aec-f1f3b4fede3d None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:17.834145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 360/708] 10.222.0.16 () {56 vars in 1133 bytes} [Mon Sep 5 12:06:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:06:17.835411 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd3155-c07a-4b69-aca4-e44e2ea86bdd None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:17.889369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:17.925129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:17.953833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602637b4-ebdc-4c9f-9aec-f1f3b4fede3d None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:17.955265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd3155-c07a-4b69-aca4-e44e2ea86bdd None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:17.991909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82c6df7-fa81-423d-8da1-9cbc92cd1e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:17.992785 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c6df7-fa81-423d-8da1-9cbc92cd1e3c None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:17.993320 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c6df7-fa81-423d-8da1-9cbc92cd1e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:17.993730 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c6df7-fa81-423d-8da1-9cbc92cd1e3c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:17.994451 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82c6df7-fa81-423d-8da1-9cbc92cd1e3c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:17.996086 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82c6df7-fa81-423d-8da1-9cbc92cd1e3c None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:17.999629 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3548f1-44fa-4c50-9358-1fdeaee21baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:18.000424 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3548f1-44fa-4c50-9358-1fdeaee21baf None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:18.000880 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3548f1-44fa-4c50-9358-1fdeaee21baf None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:18.001326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3548f1-44fa-4c50-9358-1fdeaee21baf None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:18.002165 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3548f1-44fa-4c50-9358-1fdeaee21baf None admin] RBAC: Adding query filter params (name=network) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:18.002642 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3548f1-44fa-4c50-9358-1fdeaee21baf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:18.004199 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3548f1-44fa-4c50-9358-1fdeaee21baf None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:18.006547 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-b82c6df7-fa81-423d-8da1-9cbc92cd1e3c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 12:06:18.006547 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:18.006547 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:06:18.006547 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:06:18.006547 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:06:18.006547 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:06:18.006547 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:06:18.006547 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:18.006547 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 12:06:18.006547 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:18.006547 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:06:18.006547 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:18.006547 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:18.006547 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:18.006547 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:18.006547 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:18.006547 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:18.007746 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:18.007746 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:18.007746 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:18.007746 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:18.007746 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:18.007746 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:18.007746 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:06:18.007746 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:06:18.007746 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:06:18.007746 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:06:18.007746 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:18.007746 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:18.007746 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:18.007746 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:18.007746 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:18.007746 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:18.008660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:18.008660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:18.008660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:18.008660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:18.008660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:18.008660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:18.008660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:18.008660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:18.008660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:18.008660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:18.008660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:06:18.008660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:18.008660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 12:06:18.008660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:18.016723 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 349/710] 10.222.0.16 () {56 vars in 1129 bytes} [Mon Sep 5 12:06:17 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:18.016723 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82c6df7-fa81-423d-8da1-9cbc92cd1e3c None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:18.017084 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 361/710] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Sep 5 12:06:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:18.018133 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3548f1-44fa-4c50-9358-1fdeaee21baf None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:18.124474 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82c6df7-fa81-423d-8da1-9cbc92cd1e3c None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:18.124803 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3548f1-44fa-4c50-9358-1fdeaee21baf None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:18.135901 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:18.166928 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50657e-3d0a-4859-8caa-c43378f404bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:18.167557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50657e-3d0a-4859-8caa-c43378f404bf None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:18.167827 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50657e-3d0a-4859-8caa-c43378f404bf None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:18.168096 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50657e-3d0a-4859-8caa-c43378f404bf None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:18.168720 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e50657e-3d0a-4859-8caa-c43378f404bf None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:18.169045 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e50657e-3d0a-4859-8caa-c43378f404bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:18.170394 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e50657e-3d0a-4859-8caa-c43378f404bf None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:18.172484 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7fcad-0b67-49af-91da-5ae25a54639d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:18.173081 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7fcad-0b67-49af-91da-5ae25a54639d None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:18.173351 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7fcad-0b67-49af-91da-5ae25a54639d None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:18.173620 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7fcad-0b67-49af-91da-5ae25a54639d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:18.182362 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 362/711] 10.222.0.16 () {56 vars in 1124 bytes} [Mon Sep 5 12:06:18 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:18.182362 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50657e-3d0a-4859-8caa-c43378f404bf None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:18.191293 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:18.260657 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc7fcad-0b67-49af-91da-5ae25a54639d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:18.262168 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc7fcad-0b67-49af-91da-5ae25a54639d None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:18.324500 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50657e-3d0a-4859-8caa-c43378f404bf None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:18.369211 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2be48b-d8e8-4bb6-967e-ef1793484274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:18.370210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2be48b-d8e8-4bb6-967e-ef1793484274 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:18.370689 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2be48b-d8e8-4bb6-967e-ef1793484274 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:18.371157 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2be48b-d8e8-4bb6-967e-ef1793484274 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:18.372009 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2be48b-d8e8-4bb6-967e-ef1793484274 None admin] RBAC: Adding query filter params (type=network) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:18.372901 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2be48b-d8e8-4bb6-967e-ef1793484274 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:18.373758 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 350/712] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:06:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:06:18.374724 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7fcad-0b67-49af-91da-5ae25a54639d None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:18.375013 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2be48b-d8e8-4bb6-967e-ef1793484274 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:18.383188 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 363/713] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Sep 5 12:06:18 2022] GET /identity/v3/services?type=network => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:18.384940 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2be48b-d8e8-4bb6-967e-ef1793484274 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:18.421007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:18.458326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:18.546121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7fcad-0b67-49af-91da-5ae25a54639d None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:18.579425 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2be48b-d8e8-4bb6-967e-ef1793484274 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:18.588975 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93774b4-2307-40be-8739-921612e703de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:18.589602 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93774b4-2307-40be-8739-921612e703de None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:18.589879 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93774b4-2307-40be-8739-921612e703de None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:18.590151 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93774b4-2307-40be-8739-921612e703de None admin] PATH_INFO: `/v3/services` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:18.590812 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93774b4-2307-40be-8739-921612e703de None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:18.591157 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93774b4-2307-40be-8739-921612e703de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:18.592642 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93774b4-2307-40be-8739-921612e703de None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:18.601302 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 351/714] 10.222.0.16 () {56 vars in 1124 bytes} [Mon Sep 5 12:06:18 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:18.627355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f99fa-4706-4b08-8181-e79ff9e8b9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:18.628054 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f99fa-4706-4b08-8181-e79ff9e8b9dd None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:18.628329 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f99fa-4706-4b08-8181-e79ff9e8b9dd None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:18.628598 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f99fa-4706-4b08-8181-e79ff9e8b9dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:18.640914 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8f99fa-4706-4b08-8181-e79ff9e8b9dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:18.642364 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8f99fa-4706-4b08-8181-e79ff9e8b9dd None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:18.658662 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-de8f99fa-4706-4b08-8181-e79ff9e8b9dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:18.658662 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:18.658662 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:18.658662 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:18.658662 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:18.658662 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:18.658662 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:18.658662 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:18.658662 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:18.658662 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:18.658662 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:18.658662 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:18.658662 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:18.658662 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:18.658662 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:18.658662 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:18.659711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:18.659711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:18.659711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:18.659711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:18.659711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:18.659711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:18.659711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:18.659711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:18.659711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:18.659711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:18.659711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:18.659711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:18.659711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:18.659711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:18.659711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:18.659711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:18.659711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:18.660616 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:18.660616 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:18.660616 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:18.660616 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 364/715] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:18.665526 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93774b4-2307-40be-8739-921612e703de None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:18.814726 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93774b4-2307-40be-8739-921612e703de None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:18.816391 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:18.852059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0d7ff-c216-4875-bd9c-de05003e6c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:18.852579 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0d7ff-c216-4875-bd9c-de05003e6c36 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:18.852855 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0d7ff-c216-4875-bd9c-de05003e6c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:18.853188 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0d7ff-c216-4875-bd9c-de05003e6c36 None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:18.853843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e0d7ff-c216-4875-bd9c-de05003e6c36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:18.854145 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e0d7ff-c216-4875-bd9c-de05003e6c36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:18.855516 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e0d7ff-c216-4875-bd9c-de05003e6c36 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:18.870389 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 352/716] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:18.876126 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f99fa-4706-4b08-8181-e79ff9e8b9dd None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:19.074186 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53950387-6e36-4b38-8df7-30f710714171 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:19.074593 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53950387-6e36-4b38-8df7-30f710714171 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:19.074773 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53950387-6e36-4b38-8df7-30f710714171 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:19.077605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 353/717] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:06:19.082378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc315d-9f88-4e5c-85f1-aa16c46ed37c None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:19.082682 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc315d-9f88-4e5c-85f1-aa16c46ed37c None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:19.082940 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc315d-9f88-4e5c-85f1-aa16c46ed37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:19.102717 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-05cc315d-9f88-4e5c-85f1-aa16c46ed37c None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:19.103934 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-05cc315d-9f88-4e5c-85f1-aa16c46ed37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:19.137082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:19.161937 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f99fa-4706-4b08-8181-e79ff9e8b9dd None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:19.226738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 354/718] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:06:19.231193 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1525e6-b8ea-4a84-860e-bca30d77689b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:19.232653 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1525e6-b8ea-4a84-860e-bca30d77689b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:19.233078 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1525e6-b8ea-4a84-860e-bca30d77689b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:19.233490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1525e6-b8ea-4a84-860e-bca30d77689b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:19.241784 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1525e6-b8ea-4a84-860e-bca30d77689b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:19.243600 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1525e6-b8ea-4a84-860e-bca30d77689b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:19.258059 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-4b1525e6-b8ea-4a84-860e-bca30d77689b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:19.258059 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:19.258059 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:19.258059 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:19.258059 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:19.258059 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:19.258059 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:19.258059 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:19.258059 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:19.258059 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:19.258059 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:19.258059 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:19.258059 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:06:19.258059 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:06:19.258059 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:06:19.259236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:06:19.259236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:19.259236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:19.259236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:19.259236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:19.259236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:19.259236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:19.259236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:19.259236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:19.259236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:19.259236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:19.259236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:19.259236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:19.259236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:19.259236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:19.259236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:19.259236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:19.260393 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:06:19.260393 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:06:19.260393 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:06:19.260393 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:06:19.260393 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:06:19.260393 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:06:19.260393 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:19.260393 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:19.262644 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 365/719] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:06:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:19.268117 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc315d-9f88-4e5c-85f1-aa16c46ed37c None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:19.384519 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:19.395030 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc315d-9f88-4e5c-85f1-aa16c46ed37c None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:19.432388 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4c66f-395a-4d2e-8f0c-ae4ea7913594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:19.433051 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4c66f-395a-4d2e-8f0c-ae4ea7913594 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:19.433395 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4c66f-395a-4d2e-8f0c-ae4ea7913594 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:19.433665 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4c66f-395a-4d2e-8f0c-ae4ea7913594 None admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:19.434415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e4c66f-395a-4d2e-8f0c-ae4ea7913594 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:19.434763 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e4c66f-395a-4d2e-8f0c-ae4ea7913594 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:19.436221 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e4c66f-395a-4d2e-8f0c-ae4ea7913594 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:19.449220 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 355/720] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:06:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:19.456366 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1525e6-b8ea-4a84-860e-bca30d77689b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:19.558390 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3130543a-0c85-4df8-8b38-18d074923416 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:19.558750 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3130543a-0c85-4df8-8b38-18d074923416 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:19.558944 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3130543a-0c85-4df8-8b38-18d074923416 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:19.560705 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 356/721] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:06:19.566005 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392b7d0-f2d1-4c53-9f07-4c67883bdda8 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:19.566355 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392b7d0-f2d1-4c53-9f07-4c67883bdda8 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:19.566638 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392b7d0-f2d1-4c53-9f07-4c67883bdda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:19.580499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1525e6-b8ea-4a84-860e-bca30d77689b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:19.604570 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-5392b7d0-f2d1-4c53-9f07-4c67883bdda8 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:19.605970 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-5392b7d0-f2d1-4c53-9f07-4c67883bdda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:19.623289 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22269298-80b4-495c-8b72-1317406bc745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:19.624272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22269298-80b4-495c-8b72-1317406bc745 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:19.624768 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22269298-80b4-495c-8b72-1317406bc745 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:19.625236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22269298-80b4-495c-8b72-1317406bc745 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:19.626166 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22269298-80b4-495c-8b72-1317406bc745 None admin] RBAC: Adding query filter params (role.id=618c48706f7f445abeddf7eb2f8e927a, scope.project.id=fa3178684d184ec880f17640d19d55b7, user.id=7cda290418504f0db85d665479a05353) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:19.626684 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22269298-80b4-495c-8b72-1317406bc745 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:19.628346 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22269298-80b4-495c-8b72-1317406bc745 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:19.639178 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 366/722] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:06:19 2022] GET /identity/v3/role_assignments?user.id=7cda290418504f0db85d665479a05353&scope.project.id=fa3178684d184ec880f17640d19d55b7&role.id=618c48706f7f445abeddf7eb2f8e927a => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:19.659828 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22269298-80b4-495c-8b72-1317406bc745 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:19.678103 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:19.704505 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 357/723] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:06:19.773579 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22269298-80b4-495c-8b72-1317406bc745 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:19.810929 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17c644-e35d-4108-96c3-851c575e8f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:19.811601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17c644-e35d-4108-96c3-851c575e8f1e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:19.811882 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17c644-e35d-4108-96c3-851c575e8f1e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:19.812153 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17c644-e35d-4108-96c3-851c575e8f1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:19.823018 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f17c644-e35d-4108-96c3-851c575e8f1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:06:19.828469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-2f17c644-e35d-4108-96c3-851c575e8f1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:06:19.828469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:19.828469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:19.828469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:19.828469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:19.828469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:19.828469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:19.828469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:19.828469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:19.828469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:19.828469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:19.828469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:19.828469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:06:19.828469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:06:19.828469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:06:19.828469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:06:19.829445 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:06:19.829445 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:06:19.829445 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:06:19.829445 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:06:19.829445 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:19.829445 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:19.829445 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:19.829445 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:19.829445 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:19.829445 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:19.829445 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:19.829445 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:19.829445 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:19.829445 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:19.829445 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:19.829445 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:19.830314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:19.830314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:19.830314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:19.830314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:19.830314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:06:19.830314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:06:19.830314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:06:19.830314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:06:19.830314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:06:19.830314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:06:19.830314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:06:19.830314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:19.830314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 367/724] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:06:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:19.837530 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392b7d0-f2d1-4c53-9f07-4c67883bdda8 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:19.938063 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:19.957759 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392b7d0-f2d1-4c53-9f07-4c67883bdda8 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:19.999275 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e37fed-a000-4982-a8b4-cf75d47e68f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:19.999941 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e37fed-a000-4982-a8b4-cf75d47e68f1 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:20.000223 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e37fed-a000-4982-a8b4-cf75d47e68f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:20.000503 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e37fed-a000-4982-a8b4-cf75d47e68f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:20.001163 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e37fed-a000-4982-a8b4-cf75d47e68f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:20.001471 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e37fed-a000-4982-a8b4-cf75d47e68f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:20.002848 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e37fed-a000-4982-a8b4-cf75d47e68f1 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:20.012458 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 358/725] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:06:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:20.018671 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17c644-e35d-4108-96c3-851c575e8f1e None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:20.055456 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e37fed-a000-4982-a8b4-cf75d47e68f1 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:20.194048 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:20.197420 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e37fed-a000-4982-a8b4-cf75d47e68f1 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:20.237186 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17c644-e35d-4108-96c3-851c575e8f1e None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:20.272169 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc3fc92-432a-4a78-a45f-52439299f84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:20.273264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3fc92-432a-4a78-a45f-52439299f84a None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:20.273722 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3fc92-432a-4a78-a45f-52439299f84a None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:20.274168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3fc92-432a-4a78-a45f-52439299f84a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:20.286169 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc3fc92-432a-4a78-a45f-52439299f84a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:20.287953 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc3fc92-432a-4a78-a45f-52439299f84a None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:20.304195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-dcc3fc92-432a-4a78-a45f-52439299f84a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:20.304195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:20.304195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:20.304195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:20.304195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:20.304195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:20.304195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:20.304195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:20.304195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:20.304195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:20.304195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:20.304195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:20.304195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:20.304195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:20.304195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:20.304195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:20.305218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:20.305218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:20.305218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:20.305218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:20.305218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:20.305218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:20.305218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:20.305218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:20.305218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:20.305218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:20.305218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:20.305218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:20.305218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:20.305218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:20.305218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:20.305218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:20.305218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:20.306137 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:20.306137 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:20.306137 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:20.308259 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 368/726] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:20.314099 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc3fc92-432a-4a78-a45f-52439299f84a None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:20.324258 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a935c64-2365-4769-8a94-a6d1aeaead6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:20.324917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a935c64-2365-4769-8a94-a6d1aeaead6a None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:20.325199 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a935c64-2365-4769-8a94-a6d1aeaead6a None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:20.325522 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a935c64-2365-4769-8a94-a6d1aeaead6a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:20.346274 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a935c64-2365-4769-8a94-a6d1aeaead6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:06:20.363629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-6a935c64-2365-4769-8a94-a6d1aeaead6a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:06:20.363629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:20.363629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:20.363629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:20.363629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:20.363629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:20.363629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:20.363629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:20.363629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:20.363629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:20.363629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:20.363629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:20.363629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:06:20.363629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:06:20.363629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:06:20.363629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:06:20.364655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:06:20.364655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:06:20.364655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:06:20.364655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:06:20.364655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:20.364655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:20.364655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:20.364655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:20.364655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:20.364655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:20.364655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:20.364655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:20.364655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:20.364655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:20.364655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:20.364655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:20.365511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:20.365511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:20.365511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:20.365511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:20.365511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:06:20.365511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:06:20.365511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:06:20.365511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:06:20.365511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:06:20.365511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:06:20.365511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:06:20.365511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:20.365511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 359/727] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Sep 5 12:06:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:06:20.373450 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a935c64-2365-4769-8a94-a6d1aeaead6a None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:20.386672 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:20.429962 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc3fc92-432a-4a78-a45f-52439299f84a None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:20.482869 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:20.520810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea1a9c-133b-4e8c-a6d5-4a801f007f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:20.522409 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea1a9c-133b-4e8c-a6d5-4a801f007f3d None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:20.523044 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea1a9c-133b-4e8c-a6d5-4a801f007f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:20.524604 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea1a9c-133b-4e8c-a6d5-4a801f007f3d None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:20.526137 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fea1a9c-133b-4e8c-a6d5-4a801f007f3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:20.527081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fea1a9c-133b-4e8c-a6d5-4a801f007f3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:20.529780 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fea1a9c-133b-4e8c-a6d5-4a801f007f3d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:20.539809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a935c64-2365-4769-8a94-a6d1aeaead6a None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:20.556846 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 369/728] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:06:20.564489 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea1a9c-133b-4e8c-a6d5-4a801f007f3d None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:20.613586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b704b-bb23-404e-a36b-e2e50e1c1cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:20.614620 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b704b-bb23-404e-a36b-e2e50e1c1cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:20.615118 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b704b-bb23-404e-a36b-e2e50e1c1cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:20.615645 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b704b-bb23-404e-a36b-e2e50e1c1cd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:20.616502 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b704b-bb23-404e-a36b-e2e50e1c1cd1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:20.617958 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b704b-bb23-404e-a36b-e2e50e1c1cd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:20.619962 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b704b-bb23-404e-a36b-e2e50e1c1cd1 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:20.641743 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 360/729] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Sep 5 12:06:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:20.657965 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b704b-bb23-404e-a36b-e2e50e1c1cd1 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:20.734205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea1a9c-133b-4e8c-a6d5-4a801f007f3d None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:20.751120 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:20.773376 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9914c0-69d8-4222-8919-332f2cbddd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:20.774021 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9914c0-69d8-4222-8919-332f2cbddd6b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:20.774299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9914c0-69d8-4222-8919-332f2cbddd6b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:20.774584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9914c0-69d8-4222-8919-332f2cbddd6b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:20.783326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9914c0-69d8-4222-8919-332f2cbddd6b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:20.785166 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9914c0-69d8-4222-8919-332f2cbddd6b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:20.795503 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b704b-bb23-404e-a36b-e2e50e1c1cd1 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:20.800305 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-ad9914c0-69d8-4222-8919-332f2cbddd6b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 05 12:06:20.800305 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:20.800305 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:20.800305 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:20.800305 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:20.800305 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:20.800305 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:20.800305 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:20.800305 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:20.800305 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:20.800305 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:20.800305 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:20.800305 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:06:20.800305 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:06:20.800305 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:06:20.800305 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:06:20.801264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:20.801264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:20.801264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:06:20.801264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:20.801264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:06:20.801264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:20.801264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:20.801264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:20.801264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:20.801264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:20.801264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:20.801264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:20.801264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:20.801264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:20.801264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:20.801264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:20.801264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:20.801264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:20.802246 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:20.802246 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:20.802246 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:06:20.802246 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:06:20.802246 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:06:20.802246 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:06:20.802246 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:06:20.802246 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:06:20.802246 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 05 12:06:20.802246 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:20.806272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 370/730] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:06:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:20.807834 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f54672-bc85-42af-890f-2838cec458ed None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:20.808118 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f54672-bc85-42af-890f-2838cec458ed None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:20.808391 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f54672-bc85-42af-890f-2838cec458ed None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:20.810502 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 371/731] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:20.811326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f54672-bc85-42af-890f-2838cec458ed None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:20.842904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca456e9-d0b7-4744-91b5-0854943a0729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:20.844389 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca456e9-d0b7-4744-91b5-0854943a0729 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:20.844843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca456e9-d0b7-4744-91b5-0854943a0729 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:20.845454 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca456e9-d0b7-4744-91b5-0854943a0729 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:20.861330 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca456e9-d0b7-4744-91b5-0854943a0729 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:20.863074 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca456e9-d0b7-4744-91b5-0854943a0729 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:20.874796 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:20.883480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-aca456e9-d0b7-4744-91b5-0854943a0729 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:20.883480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:20.883480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:20.883480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:20.883480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:20.883480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:20.883480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:20.883480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:20.883480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:20.883480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:20.883480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:20.883480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:20.883480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:20.883480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:20.883480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:20.883480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:20.884751 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:20.884751 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:20.884751 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:20.884751 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:20.884751 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:20.884751 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:20.884751 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:20.884751 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:20.884751 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:20.884751 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:20.884751 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:20.884751 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:20.884751 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:20.884751 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:20.884751 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:20.884751 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:20.884751 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:20.885761 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:20.885761 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:20.885761 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:20.889098 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 361/732] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:06:20.890784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56d5a0-cba7-4cf0-9c45-16a0f9a0c68d None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:20.891603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56d5a0-cba7-4cf0-9c45-16a0f9a0c68d None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:20.892088 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56d5a0-cba7-4cf0-9c45-16a0f9a0c68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:20.919547 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-6c56d5a0-cba7-4cf0-9c45-16a0f9a0c68d None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:20.921595 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-6c56d5a0-cba7-4cf0-9c45-16a0f9a0c68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:20.949562 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f54672-bc85-42af-890f-2838cec458ed None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:20.995647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d3da6-83ee-4950-bf02-6012b7495aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:20.996512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d3da6-83ee-4950-bf02-6012b7495aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:20.996841 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d3da6-83ee-4950-bf02-6012b7495aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:20.998179 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d3da6-83ee-4950-bf02-6012b7495aa1 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:20.999084 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d3da6-83ee-4950-bf02-6012b7495aa1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:20.999587 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d3da6-83ee-4950-bf02-6012b7495aa1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:21.001348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d3da6-83ee-4950-bf02-6012b7495aa1 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:21.031725 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 372/733] 10.222.0.16 () {56 vars in 1135 bytes} [Mon Sep 5 12:06:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:21.032705 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d3da6-83ee-4950-bf02-6012b7495aa1 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:21.038507 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 362/734] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:06:21.038507 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56d5a0-cba7-4cf0-9c45-16a0f9a0c68d None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:21.110091 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:21.169979 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56d5a0-cba7-4cf0-9c45-16a0f9a0c68d None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:21.181777 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d3da6-83ee-4950-bf02-6012b7495aa1 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:21.185720 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:21.212985 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc67b9c-e3ae-4239-979e-7c9890407b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:21.213837 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc67b9c-e3ae-4239-979e-7c9890407b6f None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:21.214369 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc67b9c-e3ae-4239-979e-7c9890407b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:21.214784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc67b9c-e3ae-4239-979e-7c9890407b6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:21.229249 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e119d-09ec-4740-b39f-9bf8275e9412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:21.230153 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e119d-09ec-4740-b39f-9bf8275e9412 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:21.230659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e119d-09ec-4740-b39f-9bf8275e9412 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:21.231022 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e119d-09ec-4740-b39f-9bf8275e9412 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:21.232308 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e119d-09ec-4740-b39f-9bf8275e9412 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:21.232703 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e119d-09ec-4740-b39f-9bf8275e9412 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:21.234894 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e119d-09ec-4740-b39f-9bf8275e9412 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:21.254463 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 373/735] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:21.255951 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e1fc5-da17-42c4-a836-63a76f451a3b None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:21.256329 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e1fc5-da17-42c4-a836-63a76f451a3b None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:21.256697 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e1fc5-da17-42c4-a836-63a76f451a3b None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:21.263318 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 374/736] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:21.263318 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e1fc5-da17-42c4-a836-63a76f451a3b None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:21.299542 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc67b9c-e3ae-4239-979e-7c9890407b6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:21.301335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc67b9c-e3ae-4239-979e-7c9890407b6f None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:21.360066 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:21.389806 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e1fc5-da17-42c4-a836-63a76f451a3b None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:21.427609 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 363/737] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:06:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:06:21.427609 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a7b00-f0ae-44a4-8a13-2f7374f36c3b None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:21.427609 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a7b00-f0ae-44a4-8a13-2f7374f36c3b None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:21.427609 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a7b00-f0ae-44a4-8a13-2f7374f36c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:21.431675 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d467f1-154f-4da3-b497-8d5bb4de3d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:21.432306 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d467f1-154f-4da3-b497-8d5bb4de3d44 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:21.432589 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d467f1-154f-4da3-b497-8d5bb4de3d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:21.432861 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d467f1-154f-4da3-b497-8d5bb4de3d44 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:21.441388 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d467f1-154f-4da3-b497-8d5bb4de3d44 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:21.442820 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d467f1-154f-4da3-b497-8d5bb4de3d44 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:21.444956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-853a7b00-f0ae-44a4-8a13-2f7374f36c3b None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:21.446371 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-853a7b00-f0ae-44a4-8a13-2f7374f36c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:21.457214 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-53d467f1-154f-4da3-b497-8d5bb4de3d44 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 12:06:21.457214 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:21.457214 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:21.457214 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:21.457214 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:21.457214 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:21.457214 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:21.457214 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:21.457214 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:21.457214 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:21.457214 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:21.457214 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:21.457214 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:06:21.457214 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:06:21.457214 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:06:21.457214 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:06:21.458226 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:21.458226 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:21.458226 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:06:21.458226 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:21.458226 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:06:21.458226 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:21.458226 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:21.458226 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:21.458226 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:21.458226 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:21.458226 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:21.458226 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:21.458226 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:21.458226 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:21.458226 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:21.458226 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:21.458226 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:21.458226 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:21.460278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:21.460278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:21.460278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:06:21.460278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:06:21.460278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:06:21.460278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:06:21.460278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:06:21.460278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:06:21.460278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 12:06:21.460278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:21.460278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 375/738] 10.222.0.16 () {56 vars in 1117 bytes} [Mon Sep 5 12:06:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:21.460278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d467f1-154f-4da3-b497-8d5bb4de3d44 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:21.488071 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:21.527900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 364/739] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:06:21.527900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a7b00-f0ae-44a4-8a13-2f7374f36c3b None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:21.595799 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d467f1-154f-4da3-b497-8d5bb4de3d44 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:21.611591 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:21.636573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a7b00-f0ae-44a4-8a13-2f7374f36c3b None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:21.644624 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b77eb-e3b1-42fa-929b-0374604f3ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:21.645225 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b77eb-e3b1-42fa-929b-0374604f3ce4 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:21.645500 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b77eb-e3b1-42fa-929b-0374604f3ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:21.645768 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b77eb-e3b1-42fa-929b-0374604f3ce4 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:21.646416 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b77eb-e3b1-42fa-929b-0374604f3ce4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:21.647853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b77eb-e3b1-42fa-929b-0374604f3ce4 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:21.675558 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 376/740] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Sep 5 12:06:21 2022] POST /identity/v3/services => generated 240 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:06:21.675558 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b77eb-e3b1-42fa-929b-0374604f3ce4 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:21.727580 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58771a50-43c7-49ca-acb0-763a2fd48b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:21.729871 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58771a50-43c7-49ca-acb0-763a2fd48b9c None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:21.730148 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58771a50-43c7-49ca-acb0-763a2fd48b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:21.730418 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58771a50-43c7-49ca-acb0-763a2fd48b9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:21.749648 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58771a50-43c7-49ca-acb0-763a2fd48b9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:21.751121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58771a50-43c7-49ca-acb0-763a2fd48b9c None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:21.783917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-58771a50-43c7-49ca-acb0-763a2fd48b9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:21.783917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:21.783917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:21.783917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:21.783917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:21.783917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:21.783917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:21.783917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:21.783917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:21.783917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:21.783917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:21.783917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:21.783917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:21.783917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:21.783917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:21.783917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:21.784966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:21.784966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:21.784966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:21.784966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:21.784966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:21.784966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:21.784966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:21.784966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:21.784966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:21.784966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:21.784966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:21.784966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:21.784966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:21.784966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:21.784966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:21.784966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:21.784966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:21.785888 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:21.785888 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:21.785888 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:21.790297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 365/741] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:21.797730 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58771a50-43c7-49ca-acb0-763a2fd48b9c None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:21.841532 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:21.865550 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b77eb-e3b1-42fa-929b-0374604f3ce4 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:21.913446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb73619-7d6f-421b-9ab9-3fd71a318b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:21.914067 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb73619-7d6f-421b-9ab9-3fd71a318b05 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:21.914345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb73619-7d6f-421b-9ab9-3fd71a318b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:21.914614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb73619-7d6f-421b-9ab9-3fd71a318b05 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:21.915339 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb73619-7d6f-421b-9ab9-3fd71a318b05 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:21.915674 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb73619-7d6f-421b-9ab9-3fd71a318b05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:21.917106 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb73619-7d6f-421b-9ab9-3fd71a318b05 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:21.929256 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58771a50-43c7-49ca-acb0-763a2fd48b9c None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:21.943702 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 377/742] 10.222.0.16 () {56 vars in 1149 bytes} [Mon Sep 5 12:06:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:21.943702 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb73619-7d6f-421b-9ab9-3fd71a318b05 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:21.960255 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:21.970933 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796857a-ee15-4347-bb07-9ae9ba17f8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:21.971570 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796857a-ee15-4347-bb07-9ae9ba17f8fa None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:21.971855 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796857a-ee15-4347-bb07-9ae9ba17f8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:21.972127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796857a-ee15-4347-bb07-9ae9ba17f8fa None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:21.972834 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1796857a-ee15-4347-bb07-9ae9ba17f8fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:21.973136 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1796857a-ee15-4347-bb07-9ae9ba17f8fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:21.974783 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1796857a-ee15-4347-bb07-9ae9ba17f8fa None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:21.992009 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 366/743] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:21.992009 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674fe90-a55e-4298-90a4-4014ff748f83 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:21.992009 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674fe90-a55e-4298-90a4-4014ff748f83 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:21.992009 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674fe90-a55e-4298-90a4-4014ff748f83 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:21.992009 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 367/744] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:21.992009 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674fe90-a55e-4298-90a4-4014ff748f83 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:22.062285 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb73619-7d6f-421b-9ab9-3fd71a318b05 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:22.099229 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6bdbd0-4338-4186-a66c-446beb39f8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:22.099887 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6bdbd0-4338-4186-a66c-446beb39f8ee None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:22.100155 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6bdbd0-4338-4186-a66c-446beb39f8ee None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:22.100422 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6bdbd0-4338-4186-a66c-446beb39f8ee None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:22.101078 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6bdbd0-4338-4186-a66c-446beb39f8ee None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:22.102443 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6bdbd0-4338-4186-a66c-446beb39f8ee None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:22.111129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674fe90-a55e-4298-90a4-4014ff748f83 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:22.119536 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 378/745] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Sep 5 12:06:21 2022] POST /identity/v3/services => generated 251 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:06:22.121029 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6bdbd0-4338-4186-a66c-446beb39f8ee None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:22.131549 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:22.157648 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934aaaf-16fb-448d-ba98-a0c103f640c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:22.158265 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934aaaf-16fb-448d-ba98-a0c103f640c4 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:22.158542 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934aaaf-16fb-448d-ba98-a0c103f640c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:22.158812 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934aaaf-16fb-448d-ba98-a0c103f640c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:22.218490 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:22.240786 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934aaaf-16fb-448d-ba98-a0c103f640c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:22.242270 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934aaaf-16fb-448d-ba98-a0c103f640c4 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:22.283070 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6bdbd0-4338-4186-a66c-446beb39f8ee None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:22.330344 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dcfa6e-df3d-4db6-975c-97d6d34fc328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:22.330721 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dcfa6e-df3d-4db6-975c-97d6d34fc328 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:22.330999 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dcfa6e-df3d-4db6-975c-97d6d34fc328 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:22.331268 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dcfa6e-df3d-4db6-975c-97d6d34fc328 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:22.342135 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dcfa6e-df3d-4db6-975c-97d6d34fc328 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:22.342763 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dcfa6e-df3d-4db6-975c-97d6d34fc328 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:22.355952 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-45dcfa6e-df3d-4db6-975c-97d6d34fc328 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:22.355952 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:22.355952 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:22.355952 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:22.355952 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:22.355952 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:22.355952 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:22.355952 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:22.355952 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:22.355952 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:22.355952 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:22.355952 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:22.355952 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:06:22.355952 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:06:22.355952 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:06:22.357152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:06:22.357152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:22.357152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:22.357152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:22.357152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:22.357152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:22.357152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:22.357152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:22.357152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:22.357152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:22.357152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:22.357152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:22.357152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:22.357152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:22.357152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:22.357152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:22.357152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:22.358315 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:06:22.358315 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:06:22.358315 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:06:22.358315 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:06:22.358315 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:06:22.358315 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:06:22.358315 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:22.358315 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:22.361853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 379/746] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:06:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:22.363169 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799bfbc-5fef-4327-8971-a9ec8bdddc51 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:22.363453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799bfbc-5fef-4327-8971-a9ec8bdddc51 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:22.363769 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799bfbc-5fef-4327-8971-a9ec8bdddc51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:22.370614 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 368/747] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:06:21 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:06:22.370614 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934aaaf-16fb-448d-ba98-a0c103f640c4 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:22.386793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-5799bfbc-5fef-4327-8971-a9ec8bdddc51 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:22.388865 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5799bfbc-5fef-4327-8971-a9ec8bdddc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:22.461064 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:22.466466 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:22.479546 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 380/748] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:06:22.479546 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799bfbc-5fef-4327-8971-a9ec8bdddc51 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:22.500458 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934aaaf-16fb-448d-ba98-a0c103f640c4 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:22.576943 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3bb1fc-4e30-4967-bfd5-ce6dd02e0b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:22.577906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bb1fc-4e30-4967-bfd5-ce6dd02e0b7c None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:22.578389 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bb1fc-4e30-4967-bfd5-ce6dd02e0b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:22.578887 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bb1fc-4e30-4967-bfd5-ce6dd02e0b7c None admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:22.579841 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3bb1fc-4e30-4967-bfd5-ce6dd02e0b7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:22.580420 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3bb1fc-4e30-4967-bfd5-ce6dd02e0b7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:22.580825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799bfbc-5fef-4327-8971-a9ec8bdddc51 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:22.582290 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3bb1fc-4e30-4967-bfd5-ce6dd02e0b7c None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:22.603823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 369/749] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:06:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:22.619902 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3bb1fc-4e30-4967-bfd5-ce6dd02e0b7c None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:22.622875 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae34198-912d-4ca5-9369-ccf4a7f24a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:22.623707 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae34198-912d-4ca5-9369-ccf4a7f24a91 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:22.624085 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae34198-912d-4ca5-9369-ccf4a7f24a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:22.624476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae34198-912d-4ca5-9369-ccf4a7f24a91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:22.638315 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae34198-912d-4ca5-9369-ccf4a7f24a91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:22.638315 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae34198-912d-4ca5-9369-ccf4a7f24a91 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:22.654041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-dae34198-912d-4ca5-9369-ccf4a7f24a91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:22.654041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:22.654041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:22.654041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:22.654041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:22.654041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:22.654041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:22.654041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:22.654041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:22.654041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:22.654041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:22.654041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:22.654041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:22.654041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:22.654041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:22.654041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:22.655281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:22.655281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:22.655281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:22.655281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:22.655281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:22.655281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:22.655281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:22.655281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:22.655281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:22.655281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:22.655281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:22.655281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:22.655281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:22.655281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:22.655281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:22.655281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:22.655281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:22.656245 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:22.656245 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:22.656245 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:22.656697 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 381/750] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:22.664607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae34198-912d-4ca5-9369-ccf4a7f24a91 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:22.759235 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3bb1fc-4e30-4967-bfd5-ce6dd02e0b7c None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:22.790396 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:22.803081 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc71d3-c42c-4df8-b3c9-7cefb83ce4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:22.804084 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc71d3-c42c-4df8-b3c9-7cefb83ce4db None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:22.804574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc71d3-c42c-4df8-b3c9-7cefb83ce4db None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:22.805047 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc71d3-c42c-4df8-b3c9-7cefb83ce4db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:22.805985 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc71d3-c42c-4df8-b3c9-7cefb83ce4db None admin] RBAC: Adding query filter params (role.id=618c48706f7f445abeddf7eb2f8e927a, scope.project.id=fa3178684d184ec880f17640d19d55b7, user.id=857eb7657e7d45938baca17f91a16901) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:22.806520 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc71d3-c42c-4df8-b3c9-7cefb83ce4db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:22.808229 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc71d3-c42c-4df8-b3c9-7cefb83ce4db None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:22.815573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 370/751] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:06:22 2022] GET /identity/v3/role_assignments?user.id=857eb7657e7d45938baca17f91a16901&scope.project.id=fa3178684d184ec880f17640d19d55b7&role.id=618c48706f7f445abeddf7eb2f8e927a => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:06:22.874365 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc71d3-c42c-4df8-b3c9-7cefb83ce4db None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:22.875130 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae34198-912d-4ca5-9369-ccf4a7f24a91 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:22.893346 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:22.914862 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c0efe-f629-4b4e-92cf-a28786df115f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:22.915516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c0efe-f629-4b4e-92cf-a28786df115f None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:22.915842 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c0efe-f629-4b4e-92cf-a28786df115f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:22.916115 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c0efe-f629-4b4e-92cf-a28786df115f None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:22.916765 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c0efe-f629-4b4e-92cf-a28786df115f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:22.917066 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c0efe-f629-4b4e-92cf-a28786df115f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:22.920814 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c0efe-f629-4b4e-92cf-a28786df115f None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:22.954165 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 382/752] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:22.977449 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c0efe-f629-4b4e-92cf-a28786df115f None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:23.078251 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc71d3-c42c-4df8-b3c9-7cefb83ce4db None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:23.147398 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb8da81-301e-4e47-91ad-1b8f647665d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:23.148418 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8da81-301e-4e47-91ad-1b8f647665d4 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:23.148895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8da81-301e-4e47-91ad-1b8f647665d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:23.149925 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8da81-301e-4e47-91ad-1b8f647665d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:23.152974 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c0efe-f629-4b4e-92cf-a28786df115f None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:23.162262 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:23.177431 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb8da81-301e-4e47-91ad-1b8f647665d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:06:23.183328 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-8bb8da81-301e-4e47-91ad-1b8f647665d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:06:23.183328 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:23.183328 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:23.183328 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:23.183328 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:23.183328 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:23.183328 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:23.183328 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:23.183328 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:23.183328 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:23.183328 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:23.183328 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:23.183328 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:06:23.183328 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:06:23.183328 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:06:23.183328 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:06:23.184364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:06:23.184364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:06:23.184364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:06:23.184364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:06:23.184364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:23.184364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:23.184364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:23.184364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:23.184364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:23.184364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:23.184364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:23.184364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:23.184364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:23.184364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:23.184364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:23.184364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:23.185364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:23.185364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:23.185364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:23.185364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:23.185364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:06:23.185364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:06:23.185364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:06:23.185364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:06:23.185364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:06:23.185364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:06:23.185364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:06:23.185364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:23.194334 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 371/753] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:06:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:06:23.200829 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb8da81-301e-4e47-91ad-1b8f647665d4 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:23.237198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd554c81-5416-423e-af95-a321991c20c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:23.239275 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd554c81-5416-423e-af95-a321991c20c9 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:23.239990 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd554c81-5416-423e-af95-a321991c20c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:23.241256 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd554c81-5416-423e-af95-a321991c20c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:23.255038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd554c81-5416-423e-af95-a321991c20c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:23.257599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd554c81-5416-423e-af95-a321991c20c9 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:23.284490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-cd554c81-5416-423e-af95-a321991c20c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:23.284490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:23.284490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:23.284490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:23.284490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:23.284490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:23.284490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:23.284490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:23.284490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:23.284490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:23.284490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:23.284490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:23.284490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:06:23.284490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:06:23.284490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:06:23.285474 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:06:23.285474 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:23.285474 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:23.285474 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:23.285474 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:23.285474 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:23.285474 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:23.285474 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:23.285474 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:23.285474 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:23.285474 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:23.285474 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:23.285474 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:23.285474 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:23.285474 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:23.285474 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:23.285474 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:23.286407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:06:23.286407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:06:23.286407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:06:23.286407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:06:23.286407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:06:23.286407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:06:23.286407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:23.286407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:23.290651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 383/754] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:06:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:23.297318 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd554c81-5416-423e-af95-a321991c20c9 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:23.335952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:23.399582 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb8da81-301e-4e47-91ad-1b8f647665d4 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:23.414210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd554c81-5416-423e-af95-a321991c20c9 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:23.451469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266a043-b9da-4550-86ef-830982e46992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:23.452111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266a043-b9da-4550-86ef-830982e46992 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:23.452394 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266a043-b9da-4550-86ef-830982e46992 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:23.452670 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266a043-b9da-4550-86ef-830982e46992 None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:23.453484 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3266a043-b9da-4550-86ef-830982e46992 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:23.453792 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3266a043-b9da-4550-86ef-830982e46992 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:23.454176 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38bac4-bb85-4db0-9fc8-5382d1b3cb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:23.455015 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38bac4-bb85-4db0-9fc8-5382d1b3cb4f None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:23.455238 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3266a043-b9da-4550-86ef-830982e46992 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:23.455728 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38bac4-bb85-4db0-9fc8-5382d1b3cb4f None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:23.456212 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38bac4-bb85-4db0-9fc8-5382d1b3cb4f None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:23.457082 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38bac4-bb85-4db0-9fc8-5382d1b3cb4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:23.457581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38bac4-bb85-4db0-9fc8-5382d1b3cb4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:23.459162 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38bac4-bb85-4db0-9fc8-5382d1b3cb4f None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:23.469399 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 384/755] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:06:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:06:23.477574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 372/756] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:06:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:23.478601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266a043-b9da-4550-86ef-830982e46992 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:23.482890 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38bac4-bb85-4db0-9fc8-5382d1b3cb4f None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:23.534474 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:23.666663 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266a043-b9da-4550-86ef-830982e46992 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:23.681519 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38bac4-bb85-4db0-9fc8-5382d1b3cb4f None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:23.722209 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfda587-0cb6-406c-abce-a1f795f45b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:23.723199 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfda587-0cb6-406c-abce-a1f795f45b4d None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:23.723809 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfda587-0cb6-406c-abce-a1f795f45b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:23.724303 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfda587-0cb6-406c-abce-a1f795f45b4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:23.725229 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfda587-0cb6-406c-abce-a1f795f45b4d None admin] RBAC: Adding query filter params (role.id=8cf764a0144a490a822fcf86d7aceeb0, scope.project.id=fa3178684d184ec880f17640d19d55b7, user.id=8c5ae98336c441118d01f46a50f732d7) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:23.725742 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfda587-0cb6-406c-abce-a1f795f45b4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:23.728725 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfda587-0cb6-406c-abce-a1f795f45b4d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:23.731887 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:23.741887 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 385/757] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:06:23 2022] GET /identity/v3/role_assignments?user.id=8c5ae98336c441118d01f46a50f732d7&scope.project.id=fa3178684d184ec880f17640d19d55b7&role.id=8cf764a0144a490a822fcf86d7aceeb0 => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:06:23.744185 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5204d0-8255-4772-8500-34380ffc956a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:23.745089 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5204d0-8255-4772-8500-34380ffc956a None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:23.745580 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5204d0-8255-4772-8500-34380ffc956a None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:23.746083 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5204d0-8255-4772-8500-34380ffc956a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:23.758916 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5204d0-8255-4772-8500-34380ffc956a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:23.760749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5204d0-8255-4772-8500-34380ffc956a None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:23.777480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-9f5204d0-8255-4772-8500-34380ffc956a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:23.777480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:23.777480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:23.777480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:23.777480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:23.777480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:23.777480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:23.777480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:23.777480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:23.777480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:23.777480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:23.777480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:23.777480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:23.777480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:23.777480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:23.777480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:23.779378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:23.779378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:23.779378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:23.779378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:23.779378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:23.779378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:23.779378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:23.779378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:23.779378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:23.779378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:23.779378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:23.779378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:23.779378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:23.779378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:23.779378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:23.779378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:23.779378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:23.780333 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:23.780333 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:23.780333 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:23.782643 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 373/758] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:23.788733 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfda587-0cb6-406c-abce-a1f795f45b4d None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:23.809517 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba8367-cdd2-47b4-a494-ba1f6cc96402 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:23.810373 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba8367-cdd2-47b4-a494-ba1f6cc96402 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:23.810614 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba8367-cdd2-47b4-a494-ba1f6cc96402 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:23.812333 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 374/759] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:06:23.818005 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2b858-8797-4844-bcfa-2085718d7c6c None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:23.818342 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2b858-8797-4844-bcfa-2085718d7c6c None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:23.818568 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2b858-8797-4844-bcfa-2085718d7c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:23.860713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-3ef2b858-8797-4844-bcfa-2085718d7c6c None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:23.861955 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-3ef2b858-8797-4844-bcfa-2085718d7c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:23.913176 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:23.933996 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfda587-0cb6-406c-abce-a1f795f45b4d None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:23.982361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fff6ec-7d41-4dd0-8525-6d37d088cb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:23.982920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fff6ec-7d41-4dd0-8525-6d37d088cb64 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:23.983196 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fff6ec-7d41-4dd0-8525-6d37d088cb64 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:23.983460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fff6ec-7d41-4dd0-8525-6d37d088cb64 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:23.984135 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fff6ec-7d41-4dd0-8525-6d37d088cb64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:23.984435 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fff6ec-7d41-4dd0-8525-6d37d088cb64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:23.987265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fff6ec-7d41-4dd0-8525-6d37d088cb64 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:24.001835 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 386/760] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:24.008215 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fff6ec-7d41-4dd0-8525-6d37d088cb64 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:24.016190 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 375/761] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:06:24.135311 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fff6ec-7d41-4dd0-8525-6d37d088cb64 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:24.180450 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6aa9c-35ca-4d60-bae8-b9a8191ee970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:24.181126 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6aa9c-35ca-4d60-bae8-b9a8191ee970 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:24.181345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6aa9c-35ca-4d60-bae8-b9a8191ee970 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:24.181610 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6aa9c-35ca-4d60-bae8-b9a8191ee970 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:24.193943 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f6aa9c-35ca-4d60-bae8-b9a8191ee970 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:24.195333 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f6aa9c-35ca-4d60-bae8-b9a8191ee970 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:24.212395 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-74f6aa9c-35ca-4d60-bae8-b9a8191ee970 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:06:24.212395 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:24.212395 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:24.212395 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:24.212395 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:24.212395 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:24.212395 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:24.212395 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:24.212395 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:24.212395 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:24.212395 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:24.212395 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:24.212395 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:06:24.212395 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:06:24.212395 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:06:24.212395 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:06:24.213440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:24.213440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:24.213440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:06:24.213440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:24.213440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:06:24.213440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:24.213440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:24.213440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:24.213440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:24.213440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:24.213440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:24.213440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:24.213440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:24.213440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:24.213440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:24.213440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:24.213440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:24.213440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:24.215053 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:24.215053 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:24.215053 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:06:24.215053 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:06:24.215053 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:06:24.215053 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:06:24.215053 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:06:24.215053 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:06:24.215053 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:06:24.215053 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:24.215053 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 387/762] 10.222.0.16 () {56 vars in 1093 bytes} [Mon Sep 5 12:06:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:24.219663 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2b858-8797-4844-bcfa-2085718d7c6c None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:24.349397 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6aa9c-35ca-4d60-bae8-b9a8191ee970 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:24.385328 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:24.408248 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2b858-8797-4844-bcfa-2085718d7c6c None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:24.425889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:24.473923 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b9030-e352-4080-a935-dccd57e455fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:24.474894 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b9030-e352-4080-a935-dccd57e455fd None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:24.475398 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b9030-e352-4080-a935-dccd57e455fd None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:24.475906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b9030-e352-4080-a935-dccd57e455fd None admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:24.477233 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b9030-e352-4080-a935-dccd57e455fd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:24.477886 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b9030-e352-4080-a935-dccd57e455fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:24.480283 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b9030-e352-4080-a935-dccd57e455fd None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:24.507463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 376/763] 10.222.0.16 () {56 vars in 1133 bytes} [Mon Sep 5 12:06:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:24.509974 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc226b-d7e2-45a9-b973-7e92e45bba0e None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:24.510682 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc226b-d7e2-45a9-b973-7e92e45bba0e None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:24.511338 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc226b-d7e2-45a9-b973-7e92e45bba0e None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:24.513962 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 377/764] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:24.515201 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc226b-d7e2-45a9-b973-7e92e45bba0e None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:24.537164 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6aa9c-35ca-4d60-bae8-b9a8191ee970 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:24.577768 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e28606-f0be-41fa-88f7-b1f816816ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:24.578206 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e28606-f0be-41fa-88f7-b1f816816ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:24.578490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e28606-f0be-41fa-88f7-b1f816816ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:24.578768 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e28606-f0be-41fa-88f7-b1f816816ad2 None admin] PATH_INFO: `/v3/services/network` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:24.579368 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e28606-f0be-41fa-88f7-b1f816816ad2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:24.580950 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e28606-f0be-41fa-88f7-b1f816816ad2 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:24.594111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-82e28606-f0be-41fa-88f7-b1f816816ad2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 12:06:24.594111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:24.594111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:06:24.594111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:06:24.594111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:06:24.594111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:06:24.594111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:06:24.594111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:24.594111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 12:06:24.594111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:24.594111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:06:24.594111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:24.594111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:24.594111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:24.594111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:24.594111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:24.594111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:24.595221 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:24.595221 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:24.595221 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:24.595221 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:24.595221 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:24.595221 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:24.595221 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:06:24.595221 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:06:24.595221 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:06:24.595221 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:06:24.595221 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:24.595221 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:24.595221 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:24.595221 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:24.595221 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:24.595221 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:24.596124 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:24.596124 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:24.596124 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:24.596124 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:24.596124 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:24.596124 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:24.596124 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:24.596124 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:24.596124 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:24.596124 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:24.596124 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:06:24.596124 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:24.596124 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 12:06:24.596124 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:24.600029 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 388/765] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:06:24 2022] GET /identity/v3/services/network => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:06:24.601556 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681d1bc-60b8-4513-875f-ea3430114036 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:24.601849 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681d1bc-60b8-4513-875f-ea3430114036 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:24.602190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681d1bc-60b8-4513-875f-ea3430114036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:24.624353 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-7681d1bc-60b8-4513-875f-ea3430114036 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:24.625648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7681d1bc-60b8-4513-875f-ea3430114036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:24.661632 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc226b-d7e2-45a9-b973-7e92e45bba0e None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:24.707637 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 389/766] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:06:24.709444 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681d1bc-60b8-4513-875f-ea3430114036 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:24.714555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:24.718342 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56441e-ae13-4480-bd8c-49b073e9e27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:24.719368 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56441e-ae13-4480-bd8c-49b073e9e27a None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:24.725310 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56441e-ae13-4480-bd8c-49b073e9e27a None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:24.725863 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56441e-ae13-4480-bd8c-49b073e9e27a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:24.748010 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a56441e-ae13-4480-bd8c-49b073e9e27a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:24.749798 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a56441e-ae13-4480-bd8c-49b073e9e27a None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:24.781978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-0a56441e-ae13-4480-bd8c-49b073e9e27a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:24.781978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:24.781978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:24.781978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:24.781978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:24.781978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:24.781978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:24.781978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:24.781978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:24.781978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:24.781978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:24.781978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:24.781978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:24.781978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:24.781978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:24.781978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:24.783037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:24.783037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:24.783037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:24.783037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:24.783037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:24.783037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:24.783037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:24.783037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:24.783037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:24.783037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:24.783037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:24.783037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:24.783037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:24.783037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:24.783037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:24.783037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:24.783037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:24.784057 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:24.784057 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:24.784057 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:24.789809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 378/767] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:24.795907 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56441e-ae13-4480-bd8c-49b073e9e27a None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:24.831904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:24.921095 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56441e-ae13-4480-bd8c-49b073e9e27a None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:24.957985 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18660fa-a3f6-4419-afb6-327ed3bbff02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:24.958885 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18660fa-a3f6-4419-afb6-327ed3bbff02 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:24.959405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18660fa-a3f6-4419-afb6-327ed3bbff02 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:24.959924 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18660fa-a3f6-4419-afb6-327ed3bbff02 None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:24.960771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18660fa-a3f6-4419-afb6-327ed3bbff02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:24.961273 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18660fa-a3f6-4419-afb6-327ed3bbff02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:24.962950 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18660fa-a3f6-4419-afb6-327ed3bbff02 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:24.977457 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 379/768] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:06:24.982751 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18660fa-a3f6-4419-afb6-327ed3bbff02 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:24.987802 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681d1bc-60b8-4513-875f-ea3430114036 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:25.057788 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea5d79-1fc4-46a7-a956-0a39102e57c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:25.059194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea5d79-1fc4-46a7-a956-0a39102e57c2 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:25.059917 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea5d79-1fc4-46a7-a956-0a39102e57c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:25.060396 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea5d79-1fc4-46a7-a956-0a39102e57c2 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:25.061388 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea5d79-1fc4-46a7-a956-0a39102e57c2 None admin] RBAC: Adding query filter params (name=network) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:25.062272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea5d79-1fc4-46a7-a956-0a39102e57c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:25.064214 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea5d79-1fc4-46a7-a956-0a39102e57c2 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:25.070782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 390/769] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Sep 5 12:06:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:25.072027 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea5d79-1fc4-46a7-a956-0a39102e57c2 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:25.116284 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:25.139733 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18660fa-a3f6-4419-afb6-327ed3bbff02 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:25.150127 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:25.180350 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69c994-382b-4b99-9bb9-8341515d3ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:25.181280 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69c994-382b-4b99-9bb9-8341515d3ebe None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:25.181707 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69c994-382b-4b99-9bb9-8341515d3ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:25.182136 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69c994-382b-4b99-9bb9-8341515d3ebe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:25.188990 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea5d79-1fc4-46a7-a956-0a39102e57c2 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:25.189825 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf69c994-382b-4b99-9bb9-8341515d3ebe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:25.191444 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf69c994-382b-4b99-9bb9-8341515d3ebe None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:25.203900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-bf69c994-382b-4b99-9bb9-8341515d3ebe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:25.203900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:25.203900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:25.203900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:25.203900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:25.203900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:25.203900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:25.203900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:25.203900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:25.203900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:25.203900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:25.203900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:25.203900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:06:25.203900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:06:25.203900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:06:25.204881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:06:25.204881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:25.204881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:25.204881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:25.204881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:25.204881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:25.204881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:25.204881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:25.204881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:25.204881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:25.204881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:25.204881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:25.204881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:25.204881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:25.204881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:25.204881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:25.204881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:25.205811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:06:25.205811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:06:25.205811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:06:25.205811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:06:25.205811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:06:25.205811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:06:25.205811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:25.205811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:25.205811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 380/770] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:06:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:06:25.205811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69c994-382b-4b99-9bb9-8341515d3ebe None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:25.229339 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b427a-dcad-4dcc-bcbc-e6336f095d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:25.229889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b427a-dcad-4dcc-bcbc-e6336f095d8f None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:25.230166 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b427a-dcad-4dcc-bcbc-e6336f095d8f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:25.230433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b427a-dcad-4dcc-bcbc-e6336f095d8f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:25.231008 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3b427a-dcad-4dcc-bcbc-e6336f095d8f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:25.232429 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3b427a-dcad-4dcc-bcbc-e6336f095d8f None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:25.240936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-ed3b427a-dcad-4dcc-bcbc-e6336f095d8f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 12:06:25.240936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:25.240936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:06:25.240936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:06:25.240936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:06:25.240936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:06:25.240936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:06:25.240936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:25.240936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 12:06:25.240936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:25.240936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:06:25.240936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:25.240936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:25.240936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:25.240936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:25.240936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:25.240936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:25.242004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:25.242004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:25.242004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:25.242004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:25.242004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:25.242004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:25.242004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:06:25.242004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:06:25.242004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:06:25.242004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:06:25.242004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:25.242004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:25.242004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:25.242004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:25.242004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:25.242004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:25.248001 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:25.248001 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:25.248001 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:25.248001 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:25.248001 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:25.248001 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:25.248001 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:25.248001 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:25.248001 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:25.248001 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:25.248001 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:06:25.248001 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:25.248001 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 12:06:25.248001 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:25.248001 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 391/771] 10.222.0.16 () {56 vars in 1129 bytes} [Mon Sep 5 12:06:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:25.248828 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3916d-d1e6-4729-ae9b-6e06feb6da5d None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:25.248828 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3916d-d1e6-4729-ae9b-6e06feb6da5d None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:25.248828 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3916d-d1e6-4729-ae9b-6e06feb6da5d None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:25.249529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 392/772] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:25.250362 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3916d-d1e6-4729-ae9b-6e06feb6da5d None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:25.331969 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69c994-382b-4b99-9bb9-8341515d3ebe None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:25.354373 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3916d-d1e6-4729-ae9b-6e06feb6da5d None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:25.374356 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222583d6-f38d-4fe7-aa0b-65755f728527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:25.374983 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222583d6-f38d-4fe7-aa0b-65755f728527 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:25.375261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222583d6-f38d-4fe7-aa0b-65755f728527 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:25.375553 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222583d6-f38d-4fe7-aa0b-65755f728527 None admin] PATH_INFO: `/v3/services` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:25.376200 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222583d6-f38d-4fe7-aa0b-65755f728527 None admin] RBAC: Adding query filter params (type=network) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:25.376504 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222583d6-f38d-4fe7-aa0b-65755f728527 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:25.377878 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222583d6-f38d-4fe7-aa0b-65755f728527 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:25.387659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 381/773] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Sep 5 12:06:25 2022] GET /identity/v3/services?type=network => generated 350 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:25.387659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222583d6-f38d-4fe7-aa0b-65755f728527 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:25.390555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:25.392305 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:25.398058 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bb62f-a346-4825-90ab-c195e6e64f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:25.398674 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bb62f-a346-4825-90ab-c195e6e64f87 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:25.398949 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bb62f-a346-4825-90ab-c195e6e64f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:25.399336 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bb62f-a346-4825-90ab-c195e6e64f87 None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:25.400059 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237bb62f-a346-4825-90ab-c195e6e64f87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:25.400356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237bb62f-a346-4825-90ab-c195e6e64f87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:25.401782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237bb62f-a346-4825-90ab-c195e6e64f87 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:25.415327 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 393/774] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:06:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:25.416490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bd0bf-91fe-4059-8659-17d70761d351 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:25.416778 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bd0bf-91fe-4059-8659-17d70761d351 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:25.417053 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bd0bf-91fe-4059-8659-17d70761d351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:25.437121 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-007bd0bf-91fe-4059-8659-17d70761d351 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:25.438956 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-007bd0bf-91fe-4059-8659-17d70761d351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:25.510921 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222583d6-f38d-4fe7-aa0b-65755f728527 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:25.516669 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 394/775] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:06:25.517349 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007bd0bf-91fe-4059-8659-17d70761d351 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:25.561789 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54faf4eb-410e-455b-bc72-dd63a230e7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:25.562773 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54faf4eb-410e-455b-bc72-dd63a230e7dc None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:25.563256 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54faf4eb-410e-455b-bc72-dd63a230e7dc None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:25.563789 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54faf4eb-410e-455b-bc72-dd63a230e7dc None admin] PATH_INFO: `/v3/services` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:25.564678 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54faf4eb-410e-455b-bc72-dd63a230e7dc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:25.565182 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54faf4eb-410e-455b-bc72-dd63a230e7dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:25.567293 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54faf4eb-410e-455b-bc72-dd63a230e7dc None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:25.574502 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 382/776] 10.222.0.16 () {56 vars in 1124 bytes} [Mon Sep 5 12:06:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:25.576348 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54faf4eb-410e-455b-bc72-dd63a230e7dc None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:25.644092 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007bd0bf-91fe-4059-8659-17d70761d351 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:25.666247 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:25.668434 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:25.684457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5930316-adda-45b5-b441-17efcad0eceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:25.685629 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5930316-adda-45b5-b441-17efcad0eceb None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:25.685914 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5930316-adda-45b5-b441-17efcad0eceb None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:25.686189 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5930316-adda-45b5-b441-17efcad0eceb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:25.686906 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5930316-adda-45b5-b441-17efcad0eceb None admin] RBAC: Adding query filter params (interface=public, service_id=0c57a859a8a24ba088c562b43f8e0885, region_id=RegionOne) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:25.687208 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5930316-adda-45b5-b441-17efcad0eceb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:25.688646 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5930316-adda-45b5-b441-17efcad0eceb None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:25.701591 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 395/777] 10.222.0.16 () {56 vars in 1251 bytes} [Mon Sep 5 12:06:25 2022] GET /identity/v3/endpoints?service_id=0c57a859a8a24ba088c562b43f8e0885&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:25.702537 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5930316-adda-45b5-b441-17efcad0eceb None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:25.714737 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54faf4eb-410e-455b-bc72-dd63a230e7dc None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:25.762185 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa31e4ab-1443-4541-b981-fa6bfc20605a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:25.763591 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa31e4ab-1443-4541-b981-fa6bfc20605a None admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:25.764082 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa31e4ab-1443-4541-b981-fa6bfc20605a None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:25.764830 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa31e4ab-1443-4541-b981-fa6bfc20605a None admin] PATH_INFO: `/v3/projects/fa3178684d184ec880f17640d19d55b7/users/7cda290418504f0db85d665479a05353/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:25.767937 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa31e4ab-1443-4541-b981-fa6bfc20605a None admin] RBAC: Authorizing `identity:create_grant(project_id=fa3178684d184ec880f17640d19d55b7, user_id=7cda290418504f0db85d665479a05353, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:25.770314 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa31e4ab-1443-4541-b981-fa6bfc20605a None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:25.790388 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 383/778] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Sep 5 12:06:25 2022] PUT /identity/v3/projects/fa3178684d184ec880f17640d19d55b7/users/7cda290418504f0db85d665479a05353/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:06:25.792028 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa31e4ab-1443-4541-b981-fa6bfc20605a None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:25.828506 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5930316-adda-45b5-b441-17efcad0eceb None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:25.871931 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22dea51-86b8-435d-b6d1-1879c10bceee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:25.873115 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22dea51-86b8-435d-b6d1-1879c10bceee None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:25.873394 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22dea51-86b8-435d-b6d1-1879c10bceee None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:25.873665 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22dea51-86b8-435d-b6d1-1879c10bceee None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:25.874324 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22dea51-86b8-435d-b6d1-1879c10bceee None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:25.874629 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22dea51-86b8-435d-b6d1-1879c10bceee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:25.876065 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22dea51-86b8-435d-b6d1-1879c10bceee None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:25.881460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 396/779] 10.222.0.16 () {56 vars in 1124 bytes} [Mon Sep 5 12:06:25 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:25.882320 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22dea51-86b8-435d-b6d1-1879c10bceee None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:25.945526 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:26.007033 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22dea51-86b8-435d-b6d1-1879c10bceee None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:26.040133 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa31e4ab-1443-4541-b981-fa6bfc20605a None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:26.043429 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d019d-eb95-4836-9bda-f43d414861ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:26.044536 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d019d-eb95-4836-9bda-f43d414861ee None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:26.044813 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d019d-eb95-4836-9bda-f43d414861ee None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:26.045106 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d019d-eb95-4836-9bda-f43d414861ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:26.056403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d019d-eb95-4836-9bda-f43d414861ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:06:26.061218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-608d019d-eb95-4836-9bda-f43d414861ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:06:26.061218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:26.061218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:26.061218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:26.061218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:26.061218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:26.061218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:26.061218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:26.061218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:26.061218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:26.061218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:26.061218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:26.061218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:06:26.061218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:06:26.061218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:06:26.061218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:06:26.062186 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:06:26.062186 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:06:26.062186 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:06:26.062186 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:06:26.062186 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:26.062186 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:26.062186 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:26.062186 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:26.062186 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:26.062186 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:26.062186 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:26.062186 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:26.062186 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:26.062186 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:26.062186 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:26.062186 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:26.063038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:26.063038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:26.063038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:26.063038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:26.063038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:06:26.063038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:06:26.063038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:06:26.063038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:06:26.063038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:06:26.063038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:06:26.063038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:06:26.063038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:26.063038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 397/780] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:06:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:26.064258 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7976bf4-4d14-44fa-8858-6265563bd5b0 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:26.064258 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7976bf4-4d14-44fa-8858-6265563bd5b0 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:26.064258 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7976bf4-4d14-44fa-8858-6265563bd5b0 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:26.067904 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 398/781] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:26.067904 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7976bf4-4d14-44fa-8858-6265563bd5b0 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:26.079711 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:26.090437 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05776b3e-1652-4b49-ad37-3ba26399c0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:26.092106 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05776b3e-1652-4b49-ad37-3ba26399c0af None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:26.092414 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05776b3e-1652-4b49-ad37-3ba26399c0af None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:26.092658 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05776b3e-1652-4b49-ad37-3ba26399c0af None admin] PATH_INFO: `/v3/services` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:26.093267 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05776b3e-1652-4b49-ad37-3ba26399c0af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:26.094642 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05776b3e-1652-4b49-ad37-3ba26399c0af None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:26.100378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 384/782] 10.222.0.16 () {56 vars in 1087 bytes} [Mon Sep 5 12:06:25 2022] GET /identity/v3/services => generated 772 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:26.109651 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05776b3e-1652-4b49-ad37-3ba26399c0af None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:26.187841 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7976bf4-4d14-44fa-8858-6265563bd5b0 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:26.199413 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:26.239618 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e9999-fb04-44f8-a218-0aba31ace738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:26.241009 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e9999-fb04-44f8-a218-0aba31ace738 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:26.241465 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e9999-fb04-44f8-a218-0aba31ace738 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:26.248629 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e9999-fb04-44f8-a218-0aba31ace738 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:26.250096 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e9999-fb04-44f8-a218-0aba31ace738 None admin] RBAC: Adding query filter params (interface=public, service_id=fd50e0a9396a46799440c861b80d9628, region_id=RegionOne) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:26.250844 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e9999-fb04-44f8-a218-0aba31ace738 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:26.253153 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e9999-fb04-44f8-a218-0aba31ace738 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:26.264427 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05776b3e-1652-4b49-ad37-3ba26399c0af None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:26.264626 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 399/783] 10.222.0.16 () {56 vars in 1251 bytes} [Mon Sep 5 12:06:26 2022] GET /identity/v3/endpoints?service_id=fd50e0a9396a46799440c861b80d9628&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:26.264971 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9fc4b2-de23-4fad-ae22-8c0475fa510f None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:26.265258 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9fc4b2-de23-4fad-ae22-8c0475fa510f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:26.266893 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9fc4b2-de23-4fad-ae22-8c0475fa510f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:26.286976 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-cb9fc4b2-de23-4fad-ae22-8c0475fa510f None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:26.288447 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-cb9fc4b2-de23-4fad-ae22-8c0475fa510f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:26.302068 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43285be5-77aa-467d-8a42-625aa72f6af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:26.302607 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43285be5-77aa-467d-8a42-625aa72f6af7 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:26.302887 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43285be5-77aa-467d-8a42-625aa72f6af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:26.303155 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43285be5-77aa-467d-8a42-625aa72f6af7 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:26.303858 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43285be5-77aa-467d-8a42-625aa72f6af7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:26.304159 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43285be5-77aa-467d-8a42-625aa72f6af7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:26.305526 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43285be5-77aa-467d-8a42-625aa72f6af7 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:26.319606 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 385/784] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:06:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:26.320570 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43285be5-77aa-467d-8a42-625aa72f6af7 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:26.368132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 400/785] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:06:26.369329 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9fc4b2-de23-4fad-ae22-8c0475fa510f None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:26.383920 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:26.430736 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43285be5-77aa-467d-8a42-625aa72f6af7 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:26.467278 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc076085-2166-46f6-b639-0ee7424ff711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:26.467814 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc076085-2166-46f6-b639-0ee7424ff711 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:26.468089 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc076085-2166-46f6-b639-0ee7424ff711 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:26.468362 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc076085-2166-46f6-b639-0ee7424ff711 None admin] PATH_INFO: `/v3/services` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:26.468975 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc076085-2166-46f6-b639-0ee7424ff711 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:26.470333 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc076085-2166-46f6-b639-0ee7424ff711 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:26.483716 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 386/786] 10.222.0.16 () {56 vars in 1087 bytes} [Mon Sep 5 12:06:26 2022] GET /identity/v3/services => generated 772 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:26.489407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9fc4b2-de23-4fad-ae22-8c0475fa510f None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:26.491664 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:26.534290 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00debdd4-db87-4b21-971d-457d1cb416fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:26.535346 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00debdd4-db87-4b21-971d-457d1cb416fd None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:26.535867 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00debdd4-db87-4b21-971d-457d1cb416fd None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:26.536358 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00debdd4-db87-4b21-971d-457d1cb416fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:26.547502 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00debdd4-db87-4b21-971d-457d1cb416fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:26.549340 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00debdd4-db87-4b21-971d-457d1cb416fd None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:26.566614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-00debdd4-db87-4b21-971d-457d1cb416fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:26.566614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:26.566614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:26.566614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:26.566614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:26.566614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:26.566614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:26.566614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:26.566614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:26.566614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:26.566614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:26.566614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:26.566614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:26.566614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:26.566614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:26.566614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:26.567682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:26.567682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:26.567682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:26.567682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:26.567682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:26.567682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:26.567682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:26.567682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:26.567682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:26.567682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:26.567682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:26.567682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:26.567682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:26.567682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:26.567682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:26.567682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:26.567682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:26.568617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:26.568617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:26.568617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:26.570847 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 401/787] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:26.579655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc076085-2166-46f6-b639-0ee7424ff711 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:26.696688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc076085-2166-46f6-b639-0ee7424ff711 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:26.699976 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00debdd4-db87-4b21-971d-457d1cb416fd None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:26.715327 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:26.734330 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da070c37-b10f-4003-8ec7-73a60d7de6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:26.734931 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da070c37-b10f-4003-8ec7-73a60d7de6b6 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:26.735205 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da070c37-b10f-4003-8ec7-73a60d7de6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:26.735476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da070c37-b10f-4003-8ec7-73a60d7de6b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:26.736153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da070c37-b10f-4003-8ec7-73a60d7de6b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:26.736574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da070c37-b10f-4003-8ec7-73a60d7de6b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:26.738041 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da070c37-b10f-4003-8ec7-73a60d7de6b6 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:26.755252 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 387/788] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:26.756692 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da070c37-b10f-4003-8ec7-73a60d7de6b6 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:26.830544 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00debdd4-db87-4b21-971d-457d1cb416fd None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:26.887445 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31b5dd-838f-4d1d-92f9-464dc051e103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:26.889071 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31b5dd-838f-4d1d-92f9-464dc051e103 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:26.889593 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31b5dd-838f-4d1d-92f9-464dc051e103 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:26.890072 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31b5dd-838f-4d1d-92f9-464dc051e103 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:26.901727 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da070c37-b10f-4003-8ec7-73a60d7de6b6 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:26.907769 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef31b5dd-838f-4d1d-92f9-464dc051e103 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:06:26.921198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-ef31b5dd-838f-4d1d-92f9-464dc051e103 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:06:26.921198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:26.921198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:26.921198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:26.921198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:26.921198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:26.921198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:26.921198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:26.921198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:26.921198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:26.921198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:26.921198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:26.921198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:06:26.921198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:06:26.921198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:06:26.921198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:06:26.922230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:06:26.922230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:06:26.922230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:06:26.922230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:06:26.922230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:26.922230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:26.922230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:26.922230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:26.922230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:26.922230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:26.922230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:26.922230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:26.922230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:26.922230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:26.922230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:26.922230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:26.923096 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:26.923096 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:26.923096 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:26.923096 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:26.923096 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:06:26.923096 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:06:26.923096 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:06:26.923096 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:06:26.923096 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:06:26.923096 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:06:26.923096 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:06:26.923096 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:26.925864 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 402/789] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Sep 5 12:06:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:06:26.931247 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31b5dd-838f-4d1d-92f9-464dc051e103 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:26.937217 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43cd27-d1cb-489c-aac5-c1008d966837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:26.937217 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43cd27-d1cb-489c-aac5-c1008d966837 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:26.937485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43cd27-d1cb-489c-aac5-c1008d966837 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:26.937742 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43cd27-d1cb-489c-aac5-c1008d966837 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:26.947729 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43cd27-d1cb-489c-aac5-c1008d966837 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:26.950926 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43cd27-d1cb-489c-aac5-c1008d966837 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:26.963185 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:26.969659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-4d43cd27-d1cb-489c-aac5-c1008d966837 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 05 12:06:26.969659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:26.969659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:26.969659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:26.969659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:26.969659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:26.969659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:26.969659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:26.969659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:26.969659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:26.969659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:26.969659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:26.969659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:06:26.969659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:06:26.969659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:06:26.969659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:06:26.970720 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:26.970720 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:26.970720 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:06:26.970720 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:26.970720 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:06:26.970720 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:26.970720 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:26.970720 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:26.970720 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:26.970720 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:26.970720 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:26.970720 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:26.970720 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:26.970720 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:26.970720 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:26.970720 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:26.970720 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:26.970720 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:26.971713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:26.971713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:26.971713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:06:26.971713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:06:26.971713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:06:26.971713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:06:26.971713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:06:26.971713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:06:26.971713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 05 12:06:26.971713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:26.971713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 388/790] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:06:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:06:26.978644 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43cd27-d1cb-489c-aac5-c1008d966837 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:27.058494 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:27.102747 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31b5dd-838f-4d1d-92f9-464dc051e103 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:27.170215 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7521d-75fb-4a98-a5c3-01cce5abf9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:27.170819 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7521d-75fb-4a98-a5c3-01cce5abf9e0 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:27.171092 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7521d-75fb-4a98-a5c3-01cce5abf9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:27.171358 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7521d-75fb-4a98-a5c3-01cce5abf9e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:27.172046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde7521d-75fb-4a98-a5c3-01cce5abf9e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:27.172343 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde7521d-75fb-4a98-a5c3-01cce5abf9e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:27.173700 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde7521d-75fb-4a98-a5c3-01cce5abf9e0 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:27.185599 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43cd27-d1cb-489c-aac5-c1008d966837 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:27.194920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 403/791] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Sep 5 12:06:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:06:27.200353 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7521d-75fb-4a98-a5c3-01cce5abf9e0 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:27.229660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f67a55-2301-4d45-bf6d-9ec90d4e52f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:27.230574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f67a55-2301-4d45-bf6d-9ec90d4e52f0 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:27.230859 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f67a55-2301-4d45-bf6d-9ec90d4e52f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:27.231132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f67a55-2301-4d45-bf6d-9ec90d4e52f0 None admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:27.232570 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f67a55-2301-4d45-bf6d-9ec90d4e52f0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:27.235546 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f67a55-2301-4d45-bf6d-9ec90d4e52f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:27.235546 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f67a55-2301-4d45-bf6d-9ec90d4e52f0 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:27.260935 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 389/792] 10.222.0.16 () {56 vars in 1135 bytes} [Mon Sep 5 12:06:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:06:27.266847 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f67a55-2301-4d45-bf6d-9ec90d4e52f0 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:27.363959 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:27.397886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7521d-75fb-4a98-a5c3-01cce5abf9e0 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:27.451859 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:27.460214 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f67a55-2301-4d45-bf6d-9ec90d4e52f0 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:27.464237 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e6411-35c8-4a01-9ba7-e2f4bad9d63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:27.464874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e6411-35c8-4a01-9ba7-e2f4bad9d63c None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:27.465148 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e6411-35c8-4a01-9ba7-e2f4bad9d63c None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:27.465415 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e6411-35c8-4a01-9ba7-e2f4bad9d63c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:27.484696 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e6411-35c8-4a01-9ba7-e2f4bad9d63c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:27.486375 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e6411-35c8-4a01-9ba7-e2f4bad9d63c None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:27.502355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-5a1e6411-35c8-4a01-9ba7-e2f4bad9d63c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:27.502355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:27.502355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:27.502355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:27.502355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:27.502355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:27.502355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:27.502355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:27.502355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:27.502355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:27.502355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:27.502355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:27.502355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:27.502355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:27.502355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:27.502355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:27.503406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:27.503406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:27.503406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:27.503406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:27.503406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:27.503406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:27.503406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:27.503406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:27.503406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:27.503406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:27.503406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:27.503406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:27.503406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:27.503406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:27.503406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:27.503406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:27.503406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:27.504367 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:27.504367 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:27.504367 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:27.508233 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 404/793] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:06:27.509348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e6411-35c8-4a01-9ba7-e2f4bad9d63c None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:27.553811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5286564-19a2-4d56-be31-525879b0b2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:27.554823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5286564-19a2-4d56-be31-525879b0b2d1 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:27.555292 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5286564-19a2-4d56-be31-525879b0b2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:27.555783 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5286564-19a2-4d56-be31-525879b0b2d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:27.584664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5286564-19a2-4d56-be31-525879b0b2d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:27.586409 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5286564-19a2-4d56-be31-525879b0b2d1 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:27.616592 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e6411-35c8-4a01-9ba7-e2f4bad9d63c None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:27.621297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-b5286564-19a2-4d56-be31-525879b0b2d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:27.621297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:27.621297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:27.621297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:27.621297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:27.621297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:27.621297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:27.621297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:27.621297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:27.621297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:27.621297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:27.621297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:27.621297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:27.621297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:27.621297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:27.621297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:27.622303 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:27.622303 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:27.622303 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:27.622303 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:27.622303 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:27.622303 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:27.622303 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:27.622303 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:27.622303 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:27.622303 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:27.622303 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:27.622303 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:27.622303 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:27.622303 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:27.622303 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:27.622303 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:27.622303 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:27.623234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:27.623234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:27.623234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:27.633169 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 390/794] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:27.640506 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5286564-19a2-4d56-be31-525879b0b2d1 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:27.643938 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:27.651881 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f01e5-dd2e-4849-b0bd-e9ba743d4368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:27.652835 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f01e5-dd2e-4849-b0bd-e9ba743d4368 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:27.653177 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f01e5-dd2e-4849-b0bd-e9ba743d4368 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:27.654041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f01e5-dd2e-4849-b0bd-e9ba743d4368 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:27.654756 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f01e5-dd2e-4849-b0bd-e9ba743d4368 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:27.655119 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f01e5-dd2e-4849-b0bd-e9ba743d4368 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:27.656572 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f01e5-dd2e-4849-b0bd-e9ba743d4368 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:27.669781 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 405/795] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:27.675199 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f01e5-dd2e-4849-b0bd-e9ba743d4368 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:27.792076 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5286564-19a2-4d56-be31-525879b0b2d1 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:27.854177 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f01e5-dd2e-4849-b0bd-e9ba743d4368 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:27.877596 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef6454-47a2-42a7-94f5-8eea8d00440e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:27.878133 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef6454-47a2-42a7-94f5-8eea8d00440e None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:27.878411 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef6454-47a2-42a7-94f5-8eea8d00440e None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:27.878684 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef6454-47a2-42a7-94f5-8eea8d00440e None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:27.879847 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ef6454-47a2-42a7-94f5-8eea8d00440e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:27.880151 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ef6454-47a2-42a7-94f5-8eea8d00440e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:27.881963 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ef6454-47a2-42a7-94f5-8eea8d00440e None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:27.909869 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9927debc-5fe6-4c8c-9da4-b230f0f9b8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:27.910442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927debc-5fe6-4c8c-9da4-b230f0f9b8a2 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:27.910718 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927debc-5fe6-4c8c-9da4-b230f0f9b8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:27.910987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927debc-5fe6-4c8c-9da4-b230f0f9b8a2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:27.912723 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 391/796] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:27.919498 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef6454-47a2-42a7-94f5-8eea8d00440e None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:27.932983 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9927debc-5fe6-4c8c-9da4-b230f0f9b8a2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:27.934526 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9927debc-5fe6-4c8c-9da4-b230f0f9b8a2 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:27.941677 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:27.957488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-9927debc-5fe6-4c8c-9da4-b230f0f9b8a2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 12:06:27.957488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:27.957488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:27.957488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:27.957488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:27.957488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:27.957488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:27.957488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:27.957488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:27.957488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:27.957488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:27.957488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:27.957488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:06:27.957488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:06:27.957488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:06:27.957488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:06:27.959793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:27.959793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:27.959793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:06:27.959793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:27.959793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:06:27.959793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:27.959793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:27.959793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:27.959793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:27.959793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:27.959793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:27.959793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:27.959793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:27.959793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:27.959793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:27.959793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:27.959793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:27.959793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:27.969162 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:27.969162 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:27.969162 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:06:27.969162 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:06:27.969162 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:06:27.969162 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:06:27.969162 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:06:27.969162 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:06:27.969162 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 12:06:27.969162 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:27.969162 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 406/797] 10.222.0.16 () {56 vars in 1117 bytes} [Mon Sep 5 12:06:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:06:27.970596 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9927debc-5fe6-4c8c-9da4-b230f0f9b8a2 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:28.063020 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:28.097697 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef6454-47a2-42a7-94f5-8eea8d00440e None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:28.136517 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f1843-b095-4e67-812f-8c318909be69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:28.137069 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f1843-b095-4e67-812f-8c318909be69 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:28.137403 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f1843-b095-4e67-812f-8c318909be69 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:28.138201 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f1843-b095-4e67-812f-8c318909be69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:28.143447 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9927debc-5fe6-4c8c-9da4-b230f0f9b8a2 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:28.145760 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f1843-b095-4e67-812f-8c318909be69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:28.147383 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f1843-b095-4e67-812f-8c318909be69 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:28.161419 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-121f1843-b095-4e67-812f-8c318909be69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:28.161419 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:28.161419 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:28.161419 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:28.161419 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:28.161419 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:28.161419 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:28.161419 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:28.161419 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:28.161419 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:28.161419 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:28.161419 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:28.161419 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:06:28.161419 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:06:28.161419 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:06:28.162378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:06:28.162378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:28.162378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:28.162378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:28.162378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:28.162378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:28.162378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:28.162378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:28.162378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:28.162378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:28.162378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:28.162378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:28.162378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:28.162378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:28.162378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:28.162378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:28.162378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:28.163299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:06:28.163299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:06:28.163299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:06:28.163299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:06:28.163299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:06:28.163299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:06:28.163299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:28.163299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:28.167375 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 392/798] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:06:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:28.168710 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994aa900-f80c-457a-8ba0-db23eb0de8a5 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:28.169059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994aa900-f80c-457a-8ba0-db23eb0de8a5 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:28.169331 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994aa900-f80c-457a-8ba0-db23eb0de8a5 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:28.175111 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 393/799] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:28.176082 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994aa900-f80c-457a-8ba0-db23eb0de8a5 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:28.192281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868dd45a-3a47-4457-8675-eba40f984c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:28.193475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868dd45a-3a47-4457-8675-eba40f984c98 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:28.194407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868dd45a-3a47-4457-8675-eba40f984c98 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:28.195030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868dd45a-3a47-4457-8675-eba40f984c98 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:28.196098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868dd45a-3a47-4457-8675-eba40f984c98 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:28.196850 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868dd45a-3a47-4457-8675-eba40f984c98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:28.199022 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868dd45a-3a47-4457-8675-eba40f984c98 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:28.228874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 407/800] 10.222.0.16 () {56 vars in 1149 bytes} [Mon Sep 5 12:06:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:28.231431 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66beb269-1098-42c2-9638-7e801c397959 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:28.231905 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66beb269-1098-42c2-9638-7e801c397959 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:28.232350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66beb269-1098-42c2-9638-7e801c397959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:28.256777 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-66beb269-1098-42c2-9638-7e801c397959 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:28.258250 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-66beb269-1098-42c2-9638-7e801c397959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:28.261015 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:28.309401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994aa900-f80c-457a-8ba0-db23eb0de8a5 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:28.345295 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 408/801] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:06:28.345295 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66beb269-1098-42c2-9638-7e801c397959 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:28.379496 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40380f-b3fb-4b8e-b627-50483835db88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:28.380224 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40380f-b3fb-4b8e-b627-50483835db88 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:28.380503 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40380f-b3fb-4b8e-b627-50483835db88 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:28.381376 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40380f-b3fb-4b8e-b627-50483835db88 None admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:28.383869 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad40380f-b3fb-4b8e-b627-50483835db88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:28.384192 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad40380f-b3fb-4b8e-b627-50483835db88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:28.386631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad40380f-b3fb-4b8e-b627-50483835db88 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:28.416759 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 394/802] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:06:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:28.418102 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40380f-b3fb-4b8e-b627-50483835db88 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:28.431670 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:28.507395 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66beb269-1098-42c2-9638-7e801c397959 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:28.554134 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541d499-76d3-4d28-a6a9-84a24ac60bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:28.555122 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541d499-76d3-4d28-a6a9-84a24ac60bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:28.555401 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541d499-76d3-4d28-a6a9-84a24ac60bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:28.555712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541d499-76d3-4d28-a6a9-84a24ac60bd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:28.564356 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40380f-b3fb-4b8e-b627-50483835db88 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:28.571934 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0541d499-76d3-4d28-a6a9-84a24ac60bd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:28.574150 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0541d499-76d3-4d28-a6a9-84a24ac60bd6 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:28.601190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-0541d499-76d3-4d28-a6a9-84a24ac60bd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:28.601190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:28.601190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:28.601190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:28.601190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:28.601190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:28.601190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:28.601190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:28.601190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:28.601190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:28.601190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:28.601190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:28.601190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:28.601190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:28.601190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:28.601190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:28.602276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:28.602276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:28.602276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:28.602276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:28.602276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:28.602276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:28.602276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:28.602276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:28.602276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:28.602276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:28.602276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:28.602276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:28.602276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:28.602276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:28.602276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:28.602276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:28.602276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:28.603207 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:28.603207 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:28.603207 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:28.603207 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 409/803] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:28.603450 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:28.604775 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84a47f-8969-431b-bbfe-40111d75d172 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:28.604775 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84a47f-8969-431b-bbfe-40111d75d172 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:28.604775 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84a47f-8969-431b-bbfe-40111d75d172 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:28.606461 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 410/804] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:28.611849 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84a47f-8969-431b-bbfe-40111d75d172 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:28.614546 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe00c99-9e1a-47ae-95b9-23f9382654b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:28.615606 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe00c99-9e1a-47ae-95b9-23f9382654b4 None admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:28.616941 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe00c99-9e1a-47ae-95b9-23f9382654b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:28.617228 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe00c99-9e1a-47ae-95b9-23f9382654b4 None admin] PATH_INFO: `/v3/projects/fa3178684d184ec880f17640d19d55b7/users/857eb7657e7d45938baca17f91a16901/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:28.619470 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe00c99-9e1a-47ae-95b9-23f9382654b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=fa3178684d184ec880f17640d19d55b7, user_id=857eb7657e7d45938baca17f91a16901, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:28.621051 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe00c99-9e1a-47ae-95b9-23f9382654b4 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:28.643734 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 395/805] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Sep 5 12:06:28 2022] PUT /identity/v3/projects/fa3178684d184ec880f17640d19d55b7/users/857eb7657e7d45938baca17f91a16901/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:06:28.645337 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035965af-60f2-4a8e-9575-d1adbfd2af8a None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:28.645630 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035965af-60f2-4a8e-9575-d1adbfd2af8a None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:28.645905 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035965af-60f2-4a8e-9575-d1adbfd2af8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:28.678364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-035965af-60f2-4a8e-9575-d1adbfd2af8a None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:28.679663 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-035965af-60f2-4a8e-9575-d1adbfd2af8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:28.757890 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:28.776282 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84a47f-8969-431b-bbfe-40111d75d172 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:28.784348 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 396/806] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:06:28.785670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035965af-60f2-4a8e-9575-d1adbfd2af8a None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:28.850230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c8b50f-a076-489f-acf3-cf9faabb5c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:28.850841 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8b50f-a076-489f-acf3-cf9faabb5c9e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:28.851116 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8b50f-a076-489f-acf3-cf9faabb5c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:28.851384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8b50f-a076-489f-acf3-cf9faabb5c9e None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:28.852062 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c8b50f-a076-489f-acf3-cf9faabb5c9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:28.853137 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c8b50f-a076-489f-acf3-cf9faabb5c9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:28.854042 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c8b50f-a076-489f-acf3-cf9faabb5c9e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:28.868280 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 411/807] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:28.869379 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ce349-c929-4599-9c42-c3e1b340d74a None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:28.869662 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ce349-c929-4599-9c42-c3e1b340d74a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:28.869934 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ce349-c929-4599-9c42-c3e1b340d74a None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:28.871229 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 412/808] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:28.880995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed848dd8-4559-41a2-b90f-9cda4f64434a None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:28.881371 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed848dd8-4559-41a2-b90f-9cda4f64434a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:28.881646 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed848dd8-4559-41a2-b90f-9cda4f64434a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:28.904500 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-ed848dd8-4559-41a2-b90f-9cda4f64434a None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:28.906198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ed848dd8-4559-41a2-b90f-9cda4f64434a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:28.915572 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035965af-60f2-4a8e-9575-d1adbfd2af8a None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:28.940527 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:28.964321 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab718da-bf9f-436b-8b65-76f3c34be5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:28.965263 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab718da-bf9f-436b-8b65-76f3c34be5a8 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:28.965553 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab718da-bf9f-436b-8b65-76f3c34be5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:28.965866 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab718da-bf9f-436b-8b65-76f3c34be5a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:28.979767 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab718da-bf9f-436b-8b65-76f3c34be5a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:06:28.987485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-bab718da-bf9f-436b-8b65-76f3c34be5a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:06:28.987485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:28.987485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:28.987485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:28.987485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:28.987485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:28.987485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:28.987485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:28.987485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:28.987485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:28.987485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:28.987485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:28.987485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:06:28.987485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:06:28.987485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:06:28.987485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:06:28.988671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:06:28.988671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:06:28.988671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:06:28.988671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:06:28.988671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:28.988671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:28.988671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:28.988671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:28.988671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:28.988671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:28.988671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:28.988671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:28.988671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:28.988671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:28.988671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:28.988671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:28.989752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:28.989752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:28.989752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:28.989752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:28.989752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:06:28.989752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:06:28.989752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:06:28.989752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:06:28.989752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:06:28.989752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:06:28.989752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:06:28.989752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:28.989752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 397/809] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:06:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:28.990622 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab718da-bf9f-436b-8b65-76f3c34be5a8 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:29.001109 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 413/810] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:06:29.002351 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed848dd8-4559-41a2-b90f-9cda4f64434a None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:29.099603 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:29.130269 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed848dd8-4559-41a2-b90f-9cda4f64434a None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:29.151088 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab718da-bf9f-436b-8b65-76f3c34be5a8 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:29.166708 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1121ed25-3db5-44ba-be8f-2e114ada2126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:29.167271 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1121ed25-3db5-44ba-be8f-2e114ada2126 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:29.167580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1121ed25-3db5-44ba-be8f-2e114ada2126 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:29.167931 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1121ed25-3db5-44ba-be8f-2e114ada2126 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:29.168605 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1121ed25-3db5-44ba-be8f-2e114ada2126 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:29.169462 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1121ed25-3db5-44ba-be8f-2e114ada2126 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:29.170857 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1121ed25-3db5-44ba-be8f-2e114ada2126 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:29.184224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 414/811] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:06:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:29.184224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1121ed25-3db5-44ba-be8f-2e114ada2126 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:29.197888 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47683a2-462d-4b24-99d3-3692689c2481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:29.198887 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47683a2-462d-4b24-99d3-3692689c2481 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:29.199397 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47683a2-462d-4b24-99d3-3692689c2481 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:29.200173 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47683a2-462d-4b24-99d3-3692689c2481 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:29.209643 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47683a2-462d-4b24-99d3-3692689c2481 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:29.211363 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47683a2-462d-4b24-99d3-3692689c2481 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:29.224153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-c47683a2-462d-4b24-99d3-3692689c2481 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:29.224153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:29.224153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:29.224153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:29.224153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:29.224153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:29.224153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:29.224153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:29.224153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:29.224153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:29.224153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:29.224153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:29.224153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:06:29.224153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:06:29.224153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:06:29.225128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:06:29.225128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:29.225128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:29.225128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:29.225128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:29.225128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:29.225128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:29.225128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:29.225128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:29.225128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:29.225128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:29.225128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:29.225128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:29.225128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:29.225128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:29.225128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:29.225128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:29.226059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:06:29.226059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:06:29.226059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:06:29.226059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:06:29.226059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:06:29.226059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:06:29.226059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:29.226059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:29.228535 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 398/812] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:06:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:29.229575 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47683a2-462d-4b24-99d3-3692689c2481 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:29.274490 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:29.289378 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1121ed25-3db5-44ba-be8f-2e114ada2126 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:29.327635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9cdcca-319e-4f4e-a154-a483bcdbb691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:29.328302 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9cdcca-319e-4f4e-a154-a483bcdbb691 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:29.328585 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9cdcca-319e-4f4e-a154-a483bcdbb691 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:29.328860 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9cdcca-319e-4f4e-a154-a483bcdbb691 None admin] PATH_INFO: `/v3/services/network` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:29.329558 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9cdcca-319e-4f4e-a154-a483bcdbb691 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:29.331203 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9cdcca-319e-4f4e-a154-a483bcdbb691 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:29.339727 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47683a2-462d-4b24-99d3-3692689c2481 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:29.341457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-db9cdcca-319e-4f4e-a154-a483bcdbb691 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 12:06:29.341457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:29.341457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:06:29.341457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:06:29.341457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:06:29.341457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:06:29.341457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:06:29.341457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:29.341457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 12:06:29.341457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:29.341457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:06:29.341457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:29.341457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:29.341457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:29.341457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:29.341457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:29.341457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:29.342436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:29.342436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:29.342436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:29.342436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:29.342436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:29.342436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:29.342436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:06:29.342436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:06:29.342436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:06:29.342436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:06:29.342436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:29.342436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:29.342436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:29.342436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:29.342436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:29.342436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:29.343304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:29.343304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:29.343304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:29.343304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:29.343304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:29.343304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:29.343304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:29.343304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:29.343304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:29.343304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:29.343304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:06:29.343304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:29.343304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 12:06:29.343304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:29.343304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 415/813] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:06:29 2022] GET /identity/v3/services/network => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:29.344143 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9cdcca-319e-4f4e-a154-a483bcdbb691 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:29.348618 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:29.386440 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067cb5e-1d09-4179-bb1b-1a9b87f021bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:29.387100 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067cb5e-1d09-4179-bb1b-1a9b87f021bf None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:29.387379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067cb5e-1d09-4179-bb1b-1a9b87f021bf None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:29.388222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067cb5e-1d09-4179-bb1b-1a9b87f021bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:29.399721 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067cb5e-1d09-4179-bb1b-1a9b87f021bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:29.401214 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067cb5e-1d09-4179-bb1b-1a9b87f021bf None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:29.417660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-c067cb5e-1d09-4179-bb1b-1a9b87f021bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:29.417660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:29.417660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:29.417660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:29.417660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:29.417660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:29.417660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:29.417660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:29.417660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:29.417660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:29.417660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:29.417660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:29.417660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:29.417660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:29.417660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:29.417660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:29.418669 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:29.418669 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:29.418669 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:29.418669 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:29.418669 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:29.418669 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:29.418669 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:29.418669 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:29.418669 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:29.418669 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:29.418669 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:29.418669 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:29.418669 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:29.418669 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:29.418669 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:29.418669 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:29.418669 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:29.424836 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:29.424836 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:29.424836 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:29.424836 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 399/814] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:29.424836 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067cb5e-1d09-4179-bb1b-1a9b87f021bf None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:29.470235 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9cdcca-319e-4f4e-a154-a483bcdbb691 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:29.506185 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c01921-b5d7-4f6d-9fb9-6422f63e07d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:29.506797 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c01921-b5d7-4f6d-9fb9-6422f63e07d5 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:29.507079 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c01921-b5d7-4f6d-9fb9-6422f63e07d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:29.507352 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c01921-b5d7-4f6d-9fb9-6422f63e07d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:29.508070 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c01921-b5d7-4f6d-9fb9-6422f63e07d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:29.508381 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c01921-b5d7-4f6d-9fb9-6422f63e07d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:29.509828 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c01921-b5d7-4f6d-9fb9-6422f63e07d5 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:29.517653 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:29.527135 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 416/815] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:06:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:29.527135 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c01921-b5d7-4f6d-9fb9-6422f63e07d5 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:29.534458 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067cb5e-1d09-4179-bb1b-1a9b87f021bf None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:29.569767 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a908c3-f54e-4ae6-a2ab-e70f9e45a1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:29.570491 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a908c3-f54e-4ae6-a2ab-e70f9e45a1ed None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:29.570669 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a908c3-f54e-4ae6-a2ab-e70f9e45a1ed None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:29.570941 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a908c3-f54e-4ae6-a2ab-e70f9e45a1ed None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:29.571576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a908c3-f54e-4ae6-a2ab-e70f9e45a1ed None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:29.572925 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a908c3-f54e-4ae6-a2ab-e70f9e45a1ed None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:29.582689 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-61a908c3-f54e-4ae6-a2ab-e70f9e45a1ed None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 12:06:29.582689 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:29.582689 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:06:29.582689 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:06:29.582689 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:06:29.582689 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:06:29.582689 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:06:29.582689 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:29.582689 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 12:06:29.582689 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:29.582689 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:06:29.582689 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:29.582689 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:29.582689 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:29.582689 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:29.582689 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:29.582689 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:29.583802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:29.583802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:29.583802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:29.583802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:29.583802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:29.583802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:29.583802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:06:29.583802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:06:29.583802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:06:29.583802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:06:29.583802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:29.583802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:29.583802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:29.583802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:29.583802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:29.583802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:29.584664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:29.584664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:29.584664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:29.584664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:29.584664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:29.584664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:29.584664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:29.584664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:29.584664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:29.584664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:29.584664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:06:29.584664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:29.584664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 12:06:29.584664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:29.584664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 400/816] 10.222.0.16 () {56 vars in 1129 bytes} [Mon Sep 5 12:06:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:29.585503 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a908c3-f54e-4ae6-a2ab-e70f9e45a1ed None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:29.599734 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:29.641945 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c01921-b5d7-4f6d-9fb9-6422f63e07d5 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:29.678844 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc1767-21e2-48ec-a734-323d903cd3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:29.679374 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc1767-21e2-48ec-a734-323d903cd3ad None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:29.679688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc1767-21e2-48ec-a734-323d903cd3ad None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:29.679962 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc1767-21e2-48ec-a734-323d903cd3ad None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:29.680586 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bc1767-21e2-48ec-a734-323d903cd3ad None admin] RBAC: Adding query filter params (name=network) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:29.680887 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bc1767-21e2-48ec-a734-323d903cd3ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:29.682250 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bc1767-21e2-48ec-a734-323d903cd3ad None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:29.691218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 417/817] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Sep 5 12:06:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:29.691218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc1767-21e2-48ec-a734-323d903cd3ad None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:29.703541 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a908c3-f54e-4ae6-a2ab-e70f9e45a1ed None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:29.741320 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36a54c0-d72b-498b-8212-ee027c7baeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:29.741932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a54c0-d72b-498b-8212-ee027c7baeae None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:29.742235 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a54c0-d72b-498b-8212-ee027c7baeae None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:29.742511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a54c0-d72b-498b-8212-ee027c7baeae None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:29.743159 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36a54c0-d72b-498b-8212-ee027c7baeae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:29.743463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36a54c0-d72b-498b-8212-ee027c7baeae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:29.744879 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36a54c0-d72b-498b-8212-ee027c7baeae None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:29.768463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 401/818] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:29.768463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36a54c0-d72b-498b-8212-ee027c7baeae None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:29.791183 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:29.796551 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc1767-21e2-48ec-a734-323d903cd3ad None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:29.825188 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:29.833418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2317d4-2589-40f8-8985-d369fddd2d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:29.834077 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2317d4-2589-40f8-8985-d369fddd2d3b None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:29.834376 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2317d4-2589-40f8-8985-d369fddd2d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:29.834633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2317d4-2589-40f8-8985-d369fddd2d3b None admin] PATH_INFO: `/v3/projects/fa3178684d184ec880f17640d19d55b7/users/8c5ae98336c441118d01f46a50f732d7/roles/8cf764a0144a490a822fcf86d7aceeb0` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:29.849946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2317d4-2589-40f8-8985-d369fddd2d3b None admin] RBAC: Authorizing `identity:create_grant(project_id=fa3178684d184ec880f17640d19d55b7, user_id=8c5ae98336c441118d01f46a50f732d7, role_id=8cf764a0144a490a822fcf86d7aceeb0)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:29.851384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2317d4-2589-40f8-8985-d369fddd2d3b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:29.875557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 418/819] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Sep 5 12:06:29 2022] PUT /identity/v3/projects/fa3178684d184ec880f17640d19d55b7/users/8c5ae98336c441118d01f46a50f732d7/roles/8cf764a0144a490a822fcf86d7aceeb0 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:06:29.875557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2317d4-2589-40f8-8985-d369fddd2d3b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:29.906100 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36a54c0-d72b-498b-8212-ee027c7baeae None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:29.953443 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a7845-9972-44ce-a4ce-cd69898b250e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:29.954102 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a7845-9972-44ce-a4ce-cd69898b250e None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:29.954449 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a7845-9972-44ce-a4ce-cd69898b250e None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:29.954784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a7845-9972-44ce-a4ce-cd69898b250e None admin] PATH_INFO: `/v3/services` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:29.955501 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a7845-9972-44ce-a4ce-cd69898b250e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:29.955912 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a7845-9972-44ce-a4ce-cd69898b250e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:29.957373 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a7845-9972-44ce-a4ce-cd69898b250e None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:29.967937 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 402/820] 10.222.0.16 () {56 vars in 1124 bytes} [Mon Sep 5 12:06:29 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:29.969143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a7845-9972-44ce-a4ce-cd69898b250e None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:30.003754 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2317d4-2589-40f8-8985-d369fddd2d3b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:30.056456 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728fbd75-b629-411b-8c0e-09378a64dd3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:30.056456 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fbd75-b629-411b-8c0e-09378a64dd3f None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:30.056456 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fbd75-b629-411b-8c0e-09378a64dd3f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:30.056456 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fbd75-b629-411b-8c0e-09378a64dd3f None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:30.056874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728fbd75-b629-411b-8c0e-09378a64dd3f None admin] RBAC: Adding query filter params (type=network) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:30.057125 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728fbd75-b629-411b-8c0e-09378a64dd3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:30.058899 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728fbd75-b629-411b-8c0e-09378a64dd3f None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:30.062686 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:30.065299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 419/821] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Sep 5 12:06:29 2022] GET /identity/v3/services?type=network => generated 350 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:30.066008 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728fbd75-b629-411b-8c0e-09378a64dd3f None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:30.092226 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a7845-9972-44ce-a4ce-cd69898b250e None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:30.121571 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:30.131004 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1df81-dd03-4f83-846b-bef3fcbb1fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:30.131725 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1df81-dd03-4f83-846b-bef3fcbb1fdb None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:30.132327 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1df81-dd03-4f83-846b-bef3fcbb1fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:30.132775 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1df81-dd03-4f83-846b-bef3fcbb1fdb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:30.141668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1df81-dd03-4f83-846b-bef3fcbb1fdb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:30.143100 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1df81-dd03-4f83-846b-bef3fcbb1fdb None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:30.156132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-81b1df81-dd03-4f83-846b-bef3fcbb1fdb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:06:30.156132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:30.156132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:30.156132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:30.156132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:30.156132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:30.156132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:30.156132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:30.156132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:30.156132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:30.156132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:30.156132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:30.156132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:06:30.156132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:06:30.156132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:06:30.156132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:06:30.157163 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:30.157163 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:30.157163 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:06:30.157163 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:30.157163 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:06:30.157163 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:30.157163 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:30.157163 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:30.157163 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:30.157163 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:30.157163 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:30.157163 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:30.157163 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:30.157163 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:30.157163 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:30.157163 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:30.157163 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:30.157163 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:30.158552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:30.158552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:30.158552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:06:30.158552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:06:30.158552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:06:30.158552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:06:30.158552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:06:30.158552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:06:30.158552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:06:30.158552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:30.158552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 403/822] 10.222.0.16 () {56 vars in 1093 bytes} [Mon Sep 5 12:06:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:30.158552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1df81-dd03-4f83-846b-bef3fcbb1fdb None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:30.186935 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728fbd75-b629-411b-8c0e-09378a64dd3f None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:30.225345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f9f62-0e21-46b8-8076-51eb4266f6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:30.226017 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f9f62-0e21-46b8-8076-51eb4266f6ed None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:30.226298 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f9f62-0e21-46b8-8076-51eb4266f6ed None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:30.226570 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f9f62-0e21-46b8-8076-51eb4266f6ed None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:30.227196 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f9f62-0e21-46b8-8076-51eb4266f6ed None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:30.227496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f9f62-0e21-46b8-8076-51eb4266f6ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:30.229088 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f9f62-0e21-46b8-8076-51eb4266f6ed None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:30.235045 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 420/823] 10.222.0.16 () {56 vars in 1124 bytes} [Mon Sep 5 12:06:30 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:30.235927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f9f62-0e21-46b8-8076-51eb4266f6ed None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:30.269655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1df81-dd03-4f83-846b-bef3fcbb1fdb None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:30.306111 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bf33f-312e-494b-891c-3d797882faa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:30.306717 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bf33f-312e-494b-891c-3d797882faa8 None admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:30.306994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bf33f-312e-494b-891c-3d797882faa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:30.307267 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bf33f-312e-494b-891c-3d797882faa8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:30.307977 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7bf33f-312e-494b-891c-3d797882faa8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:30.309348 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7bf33f-312e-494b-891c-3d797882faa8 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:30.318756 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:30.343003 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 404/824] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:06:30 2022] POST /identity/v3/endpoints => generated 337 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:06:30.343003 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bf33f-312e-494b-891c-3d797882faa8 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:30.396798 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f9f62-0e21-46b8-8076-51eb4266f6ed None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:30.413088 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:30.482909 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e549db9d-2423-4537-9a0a-850c80b5fb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:30.483464 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549db9d-2423-4537-9a0a-850c80b5fb5b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:30.483770 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549db9d-2423-4537-9a0a-850c80b5fb5b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:30.484047 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549db9d-2423-4537-9a0a-850c80b5fb5b None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:30.484840 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e549db9d-2423-4537-9a0a-850c80b5fb5b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:30.485131 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e549db9d-2423-4537-9a0a-850c80b5fb5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:30.486638 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e549db9d-2423-4537-9a0a-850c80b5fb5b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:30.523484 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 421/825] 10.222.0.16 () {56 vars in 1133 bytes} [Mon Sep 5 12:06:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:30.532698 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e549db9d-2423-4537-9a0a-850c80b5fb5b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:30.534037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bf33f-312e-494b-891c-3d797882faa8 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:30.599423 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f2720-e2bd-4dcc-9770-b82454c5f29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:30.600847 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f2720-e2bd-4dcc-9770-b82454c5f29d None admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:30.601332 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f2720-e2bd-4dcc-9770-b82454c5f29d None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:30.601781 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f2720-e2bd-4dcc-9770-b82454c5f29d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:30.602658 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f2720-e2bd-4dcc-9770-b82454c5f29d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:30.605162 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f2720-e2bd-4dcc-9770-b82454c5f29d None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:30.637177 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 405/826] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:06:30 2022] POST /identity/v3/endpoints => generated 346 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:06:30.692941 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e549db9d-2423-4537-9a0a-850c80b5fb5b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:30.743488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3558b2-2193-4d61-912b-836603e3cfed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:30.744434 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3558b2-2193-4d61-912b-836603e3cfed None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:30.744893 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3558b2-2193-4d61-912b-836603e3cfed None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:30.745340 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3558b2-2193-4d61-912b-836603e3cfed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:30.776911 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:30.794202 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ca67f-51e2-45f1-824f-7c9fa16c6d80 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:30.794483 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ca67f-51e2-45f1-824f-7c9fa16c6d80 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:30.794754 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ca67f-51e2-45f1-824f-7c9fa16c6d80 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:30.797605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 406/827] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:06:30.802256 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a7dac-78ef-4acb-818f-7db405ef081b None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:30.802536 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a7dac-78ef-4acb-818f-7db405ef081b None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:30.802808 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a7dac-78ef-4acb-818f-7db405ef081b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:30.827142 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-ee7a7dac-78ef-4acb-818f-7db405ef081b None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:30.828421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-ee7a7dac-78ef-4acb-818f-7db405ef081b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:30.829329 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3558b2-2193-4d61-912b-836603e3cfed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:30.831019 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3558b2-2193-4d61-912b-836603e3cfed None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:30.889904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:30.931645 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 407/828] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:06:30.955927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 422/829] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:06:30 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:06:30.961582 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a7dac-78ef-4acb-818f-7db405ef081b None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:31.068694 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a7dac-78ef-4acb-818f-7db405ef081b None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:31.104352 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ae1dc-1f42-46a7-a46d-71513c37275f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:31.104982 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ae1dc-1f42-46a7-a46d-71513c37275f None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:31.105262 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ae1dc-1f42-46a7-a46d-71513c37275f None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:31.105537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ae1dc-1f42-46a7-a46d-71513c37275f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:31.116755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ae1dc-1f42-46a7-a46d-71513c37275f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:31.118204 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ae1dc-1f42-46a7-a46d-71513c37275f None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:31.134304 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-a06ae1dc-1f42-46a7-a46d-71513c37275f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:31.134304 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:31.134304 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:31.134304 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:31.134304 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:31.134304 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:31.134304 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:31.134304 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:31.134304 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:31.134304 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:31.134304 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:31.134304 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:31.134304 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:31.134304 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:31.134304 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:31.134304 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:31.135377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:31.135377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:31.135377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:31.135377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:31.135377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:31.135377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:31.135377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:31.135377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:31.135377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:31.135377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:31.135377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:31.135377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:31.135377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:31.135377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:31.135377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:31.135377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:31.135377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:31.136854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:31.136854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:31.136854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:31.139776 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 408/830] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:31.141345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3558b2-2193-4d61-912b-836603e3cfed None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:31.200993 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:31.209523 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ae1dc-1f42-46a7-a46d-71513c37275f None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:31.251099 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3558b2-2193-4d61-912b-836603e3cfed None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:31.255893 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:31.289563 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e76c4f-4dbd-4d68-92c8-a403a3fee617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:31.290577 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e76c4f-4dbd-4d68-92c8-a403a3fee617 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:31.291121 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e76c4f-4dbd-4d68-92c8-a403a3fee617 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:31.291651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e76c4f-4dbd-4d68-92c8-a403a3fee617 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:31.292500 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e76c4f-4dbd-4d68-92c8-a403a3fee617 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:31.293020 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e76c4f-4dbd-4d68-92c8-a403a3fee617 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:31.294618 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e76c4f-4dbd-4d68-92c8-a403a3fee617 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:31.315799 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 423/831] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:31.317205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e76c4f-4dbd-4d68-92c8-a403a3fee617 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:31.339169 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ae1dc-1f42-46a7-a46d-71513c37275f None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:31.381277 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527cdda1-ba05-42be-aaaf-ba4c8c0658d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:31.382138 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cdda1-ba05-42be-aaaf-ba4c8c0658d4 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:31.382511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cdda1-ba05-42be-aaaf-ba4c8c0658d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:31.382882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cdda1-ba05-42be-aaaf-ba4c8c0658d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:31.395444 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527cdda1-ba05-42be-aaaf-ba4c8c0658d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:06:31.401379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-527cdda1-ba05-42be-aaaf-ba4c8c0658d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:06:31.401379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:31.401379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:31.401379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:31.401379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:31.401379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:31.401379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:31.401379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:31.401379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:31.401379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:31.401379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:31.401379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:31.401379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:06:31.401379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:06:31.401379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:06:31.401379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:06:31.402868 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:06:31.402868 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:06:31.402868 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:06:31.402868 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:06:31.402868 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:31.402868 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:31.402868 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:31.402868 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:31.402868 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:31.402868 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:31.402868 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:31.402868 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:31.402868 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:31.402868 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:31.402868 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:31.402868 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:31.403933 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:31.403933 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:31.403933 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:31.403933 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:31.403933 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:06:31.403933 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:06:31.403933 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:06:31.403933 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:06:31.403933 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:06:31.403933 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:06:31.403933 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:06:31.403933 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:31.409353 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 409/832] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:06:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:31.410831 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527cdda1-ba05-42be-aaaf-ba4c8c0658d4 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:31.465083 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e76c4f-4dbd-4d68-92c8-a403a3fee617 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:31.481855 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:31.502051 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2bb894-c969-4e1e-aad1-2951a5b1698b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:31.502683 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bb894-c969-4e1e-aad1-2951a5b1698b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:31.502962 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bb894-c969-4e1e-aad1-2951a5b1698b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:31.503234 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bb894-c969-4e1e-aad1-2951a5b1698b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:31.510296 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2bb894-c969-4e1e-aad1-2951a5b1698b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:31.511778 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2bb894-c969-4e1e-aad1-2951a5b1698b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:31.524102 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-0d2bb894-c969-4e1e-aad1-2951a5b1698b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:31.524102 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:31.524102 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:31.524102 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:31.524102 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:31.524102 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:31.524102 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:31.524102 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:31.524102 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:31.524102 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:31.524102 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:31.524102 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:31.524102 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:06:31.524102 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:06:31.524102 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:06:31.525114 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:06:31.525114 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:31.525114 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:31.525114 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:31.525114 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:31.525114 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:31.525114 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:31.525114 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:31.525114 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:31.525114 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:31.525114 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:31.525114 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:31.525114 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:31.525114 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:31.525114 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:31.525114 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:31.525114 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:31.526013 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:06:31.526013 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:06:31.526013 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:06:31.526013 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:06:31.526013 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:06:31.526013 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:06:31.526013 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:31.526013 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:31.529590 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:31.530382 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 424/833] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:06:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:31.531809 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2bb894-c969-4e1e-aad1-2951a5b1698b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:31.540908 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527cdda1-ba05-42be-aaaf-ba4c8c0658d4 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:31.578193 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22422849-2f17-4b89-8ff2-406753fd2d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:31.579172 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22422849-2f17-4b89-8ff2-406753fd2d3a None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:31.579714 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22422849-2f17-4b89-8ff2-406753fd2d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:31.580187 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22422849-2f17-4b89-8ff2-406753fd2d3a None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:31.581104 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22422849-2f17-4b89-8ff2-406753fd2d3a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:31.581609 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22422849-2f17-4b89-8ff2-406753fd2d3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:31.583189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22422849-2f17-4b89-8ff2-406753fd2d3a None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:31.592947 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 410/834] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:06:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:31.600032 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22422849-2f17-4b89-8ff2-406753fd2d3a None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:31.640921 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2bb894-c969-4e1e-aad1-2951a5b1698b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:31.679999 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867b14c-fd97-4023-9d27-abc6bf31e8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:31.680652 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867b14c-fd97-4023-9d27-abc6bf31e8c1 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:31.680925 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867b14c-fd97-4023-9d27-abc6bf31e8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:31.681246 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867b14c-fd97-4023-9d27-abc6bf31e8c1 None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:31.681946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867b14c-fd97-4023-9d27-abc6bf31e8c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:31.682250 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867b14c-fd97-4023-9d27-abc6bf31e8c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:31.684187 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867b14c-fd97-4023-9d27-abc6bf31e8c1 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:31.703615 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 425/835] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:06:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:31.707469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867b14c-fd97-4023-9d27-abc6bf31e8c1 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:31.715797 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22422849-2f17-4b89-8ff2-406753fd2d3a None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:31.756387 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:31.762551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5e97a-cae8-4881-b47e-9de145184ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:31.763209 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5e97a-cae8-4881-b47e-9de145184ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:31.763490 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5e97a-cae8-4881-b47e-9de145184ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:31.763856 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5e97a-cae8-4881-b47e-9de145184ff0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:31.777158 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add5e97a-cae8-4881-b47e-9de145184ff0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:31.778774 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add5e97a-cae8-4881-b47e-9de145184ff0 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:31.795700 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-add5e97a-cae8-4881-b47e-9de145184ff0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:31.795700 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:31.795700 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:31.795700 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:31.795700 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:31.795700 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:31.795700 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:31.795700 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:31.795700 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:31.795700 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:31.795700 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:31.795700 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:31.795700 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:31.795700 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:31.795700 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:31.795700 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:31.796915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:31.796915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:31.796915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:31.796915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:31.796915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:31.796915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:31.796915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:31.796915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:31.796915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:31.796915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:31.796915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:31.796915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:31.796915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:31.796915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:31.796915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:31.796915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:31.796915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:31.798197 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:31.798197 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:31.798197 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:31.798197 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 411/836] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:31.802607 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5e97a-cae8-4881-b47e-9de145184ff0 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:31.812880 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:31.843572 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867b14c-fd97-4023-9d27-abc6bf31e8c1 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:31.889825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c3ae3-22e0-4e6c-b644-62025e052787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:31.890948 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c3ae3-22e0-4e6c-b644-62025e052787 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:31.891805 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c3ae3-22e0-4e6c-b644-62025e052787 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:31.892399 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c3ae3-22e0-4e6c-b644-62025e052787 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:31.893430 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c3ae3-22e0-4e6c-b644-62025e052787 None admin] RBAC: Adding query filter params (role.id=618c48706f7f445abeddf7eb2f8e927a, scope.project.id=fa3178684d184ec880f17640d19d55b7, user.id=7cda290418504f0db85d665479a05353) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:31.894100 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c3ae3-22e0-4e6c-b644-62025e052787 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:31.896063 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c3ae3-22e0-4e6c-b644-62025e052787 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:31.911553 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 426/837] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:06:31 2022] GET /identity/v3/role_assignments?user.id=7cda290418504f0db85d665479a05353&scope.project.id=fa3178684d184ec880f17640d19d55b7&role.id=618c48706f7f445abeddf7eb2f8e927a => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:31.918493 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521390ae-5fbf-40b3-97b6-2c84dbed64b7 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:31.919415 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521390ae-5fbf-40b3-97b6-2c84dbed64b7 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:31.919748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521390ae-5fbf-40b3-97b6-2c84dbed64b7 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:31.921359 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 427/838] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:06:31.925690 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5e97a-cae8-4881-b47e-9de145184ff0 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:31.927904 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affe9d3-8450-4a05-b90e-c9435b42fe11 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:31.928185 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affe9d3-8450-4a05-b90e-c9435b42fe11 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:31.928454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affe9d3-8450-4a05-b90e-c9435b42fe11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:31.952476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-5affe9d3-8450-4a05-b90e-c9435b42fe11 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:31.953702 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5affe9d3-8450-4a05-b90e-c9435b42fe11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:31.972383 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ac4895-7c60-4fb4-a5e0-1b56bbe62dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:31.973102 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac4895-7c60-4fb4-a5e0-1b56bbe62dcf None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:31.973457 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac4895-7c60-4fb4-a5e0-1b56bbe62dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:31.973806 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac4895-7c60-4fb4-a5e0-1b56bbe62dcf None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:31.974601 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ac4895-7c60-4fb4-a5e0-1b56bbe62dcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:31.975014 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ac4895-7c60-4fb4-a5e0-1b56bbe62dcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:31.976709 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ac4895-7c60-4fb4-a5e0-1b56bbe62dcf None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:31.998030 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 412/839] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:31.999749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ac4895-7c60-4fb4-a5e0-1b56bbe62dcf None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:32.060892 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 428/840] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:31 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:06:32.086435 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:32.125639 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ac4895-7c60-4fb4-a5e0-1b56bbe62dcf None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:32.162438 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13669d-c5d4-4306-bc3c-7496eca25cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:32.163118 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13669d-c5d4-4306-bc3c-7496eca25cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:32.163396 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13669d-c5d4-4306-bc3c-7496eca25cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:32.163694 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13669d-c5d4-4306-bc3c-7496eca25cd9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:32.173326 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13669d-c5d4-4306-bc3c-7496eca25cd9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:32.174821 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13669d-c5d4-4306-bc3c-7496eca25cd9 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:32.187186 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-dd13669d-c5d4-4306-bc3c-7496eca25cd9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 05 12:06:32.187186 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:32.187186 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:32.187186 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:32.187186 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:32.187186 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:32.187186 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:32.187186 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:32.187186 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:32.187186 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:32.187186 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:32.187186 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:32.187186 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:06:32.187186 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:06:32.187186 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:06:32.187186 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:06:32.188285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:32.188285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:32.188285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:06:32.188285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:32.188285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:06:32.188285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:32.188285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:32.188285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:32.188285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:32.188285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:32.188285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:32.188285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:32.188285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:32.188285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:32.188285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:32.188285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:32.188285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:32.188285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:32.189323 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:32.189323 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:32.189323 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:06:32.189323 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:06:32.189323 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:06:32.189323 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:06:32.189323 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:06:32.189323 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:06:32.189323 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 05 12:06:32.189323 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:32.189323 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 413/841] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:06:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:32.196291 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5affe9d3-8450-4a05-b90e-c9435b42fe11 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:32.227988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:32.302483 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5affe9d3-8450-4a05-b90e-c9435b42fe11 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:32.336569 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13669d-c5d4-4306-bc3c-7496eca25cd9 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:32.338545 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085b027-7723-4089-a668-df7f69df8f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:32.339331 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085b027-7723-4089-a668-df7f69df8f65 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:32.339464 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085b027-7723-4089-a668-df7f69df8f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:32.339767 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085b027-7723-4089-a668-df7f69df8f65 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:32.340508 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b085b027-7723-4089-a668-df7f69df8f65 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:32.341372 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b085b027-7723-4089-a668-df7f69df8f65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:32.342835 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b085b027-7723-4089-a668-df7f69df8f65 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:32.362123 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 429/842] 10.222.0.16 () {56 vars in 1135 bytes} [Mon Sep 5 12:06:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:32.370400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085b027-7723-4089-a668-df7f69df8f65 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:32.467281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:32.478149 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13669d-c5d4-4306-bc3c-7496eca25cd9 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:32.480666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085b027-7723-4089-a668-df7f69df8f65 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:32.512013 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:32.522498 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505578d-141b-4499-9fad-34f7cc68e972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:32.523131 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505578d-141b-4499-9fad-34f7cc68e972 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:32.523408 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505578d-141b-4499-9fad-34f7cc68e972 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:32.523775 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505578d-141b-4499-9fad-34f7cc68e972 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:32.525595 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be92e9-58a8-44a3-a634-51ab23c0eb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:32.526558 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be92e9-58a8-44a3-a634-51ab23c0eb49 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:32.527322 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be92e9-58a8-44a3-a634-51ab23c0eb49 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:32.528067 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be92e9-58a8-44a3-a634-51ab23c0eb49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:32.535500 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0505578d-141b-4499-9fad-34f7cc68e972 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:06:32.541021 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-0505578d-141b-4499-9fad-34f7cc68e972 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:06:32.541021 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:32.541021 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:32.541021 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:32.541021 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:32.541021 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:32.541021 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:32.541021 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:32.541021 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:32.541021 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:32.541021 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:32.541021 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:32.541021 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:06:32.541021 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:06:32.541021 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:06:32.541021 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:06:32.542029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:06:32.542029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:06:32.542029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:06:32.542029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:06:32.542029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:32.542029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:32.542029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:32.542029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:32.542029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:32.542029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:32.542029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:32.542029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:32.542029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:32.542029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:32.542029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:32.542029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:32.543455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:32.543455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:32.543455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:32.543455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:32.543455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:06:32.543455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:06:32.543455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:06:32.543455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:06:32.543455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:06:32.543455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:06:32.543455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:06:32.543455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:32.543455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 414/843] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Sep 5 12:06:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:06:32.548631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505578d-141b-4499-9fad-34f7cc68e972 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:32.618917 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8be92e9-58a8-44a3-a634-51ab23c0eb49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:32.620990 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8be92e9-58a8-44a3-a634-51ab23c0eb49 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:32.653429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505578d-141b-4499-9fad-34f7cc68e972 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:32.690188 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad86f87-491b-4c80-b183-52a0ffb1866f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:32.690820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad86f87-491b-4c80-b183-52a0ffb1866f None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:32.691072 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad86f87-491b-4c80-b183-52a0ffb1866f None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:32.691346 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad86f87-491b-4c80-b183-52a0ffb1866f None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:32.692050 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad86f87-491b-4c80-b183-52a0ffb1866f None admin] RBAC: Adding query filter params (name=service) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:32.692353 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad86f87-491b-4c80-b183-52a0ffb1866f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:32.693716 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad86f87-491b-4c80-b183-52a0ffb1866f None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:32.702561 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 415/844] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Sep 5 12:06:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:06:32.707346 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f89f74-f2bf-4f00-a97e-9fcd57ee0302 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:32.707683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f89f74-f2bf-4f00-a97e-9fcd57ee0302 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:32.707963 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f89f74-f2bf-4f00-a97e-9fcd57ee0302 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:32.709462 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 416/845] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:32.710010 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f89f74-f2bf-4f00-a97e-9fcd57ee0302 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:32.716300 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:32.732913 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 430/846] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:06:32 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 05 12:06:32.733911 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c5cea-804d-401d-905b-f404eaec093d None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:32.734196 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c5cea-804d-401d-905b-f404eaec093d None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:32.734470 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c5cea-804d-401d-905b-f404eaec093d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:32.752238 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-b05c5cea-804d-401d-905b-f404eaec093d None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:32.753431 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b05c5cea-804d-401d-905b-f404eaec093d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:32.767869 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:32.824640 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 431/847] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:32 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:06:32.825965 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c5cea-804d-401d-905b-f404eaec093d None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:32.836749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f89f74-f2bf-4f00-a97e-9fcd57ee0302 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:32.875034 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343e9e5-669e-4bbe-9bd1-40141bc1bdcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:32.875874 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343e9e5-669e-4bbe-9bd1-40141bc1bdcc None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:32.876252 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343e9e5-669e-4bbe-9bd1-40141bc1bdcc None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:32.876626 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343e9e5-669e-4bbe-9bd1-40141bc1bdcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:32.891000 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c343e9e5-669e-4bbe-9bd1-40141bc1bdcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:32.892545 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c343e9e5-669e-4bbe-9bd1-40141bc1bdcc None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:32.911143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-c343e9e5-669e-4bbe-9bd1-40141bc1bdcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:32.911143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:32.911143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:32.911143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:32.911143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:32.911143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:32.911143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:32.911143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:32.911143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:32.911143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:32.911143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:32.911143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:32.911143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:32.911143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:32.911143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:32.911143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:32.912227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:32.912227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:32.912227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:32.912227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:32.912227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:32.912227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:32.912227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:32.912227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:32.912227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:32.912227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:32.912227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:32.912227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:32.912227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:32.912227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:32.912227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:32.912227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:32.912227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:32.913178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:32.913178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:32.913178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:32.917525 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 417/848] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:32.918482 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343e9e5-669e-4bbe-9bd1-40141bc1bdcc None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:32.963545 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c5cea-804d-401d-905b-f404eaec093d None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:32.984904 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:33.005913 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b572650-2dc8-4f6f-8a66-daae96ea2dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:33.005913 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b572650-2dc8-4f6f-8a66-daae96ea2dee None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:33.006188 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b572650-2dc8-4f6f-8a66-daae96ea2dee None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:33.006263 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b572650-2dc8-4f6f-8a66-daae96ea2dee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:33.017613 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b572650-2dc8-4f6f-8a66-daae96ea2dee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:33.019072 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b572650-2dc8-4f6f-8a66-daae96ea2dee None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:33.035822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-6b572650-2dc8-4f6f-8a66-daae96ea2dee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:33.035822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:33.035822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:33.035822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:33.035822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:33.035822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:33.035822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:33.035822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:33.035822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:33.035822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:33.035822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:33.035822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:33.035822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:33.035822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:33.035822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:33.035822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:33.036874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:33.036874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:33.036874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:33.036874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:33.036874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:33.036874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:33.036874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:33.036874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:33.036874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:33.036874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:33.036874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:33.036874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:33.036874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:33.036874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:33.036874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:33.036874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:33.036874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:33.037781 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:33.037781 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:33.037781 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:33.037781 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 432/849] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:33.042508 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b572650-2dc8-4f6f-8a66-daae96ea2dee None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:33.059962 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:33.062299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343e9e5-669e-4bbe-9bd1-40141bc1bdcc None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:33.098558 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b4923-e9ac-4c0b-b4ad-d48c210ed1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:33.099701 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b4923-e9ac-4c0b-b4ad-d48c210ed1ed None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:33.100218 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b4923-e9ac-4c0b-b4ad-d48c210ed1ed None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:33.100685 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b4923-e9ac-4c0b-b4ad-d48c210ed1ed None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:33.101609 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b4923-e9ac-4c0b-b4ad-d48c210ed1ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:33.101910 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b4923-e9ac-4c0b-b4ad-d48c210ed1ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:33.103263 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b4923-e9ac-4c0b-b4ad-d48c210ed1ed None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:33.119587 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 418/850] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:33.119587 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b4923-e9ac-4c0b-b4ad-d48c210ed1ed None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:33.163252 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b572650-2dc8-4f6f-8a66-daae96ea2dee None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:33.196670 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4764375-e1cb-4e85-9f66-4a087de2a863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:33.197282 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4764375-e1cb-4e85-9f66-4a087de2a863 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:33.197564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4764375-e1cb-4e85-9f66-4a087de2a863 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:33.197840 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4764375-e1cb-4e85-9f66-4a087de2a863 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:33.198478 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4764375-e1cb-4e85-9f66-4a087de2a863 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:33.198783 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4764375-e1cb-4e85-9f66-4a087de2a863 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:33.200705 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4764375-e1cb-4e85-9f66-4a087de2a863 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:33.212676 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 433/851] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:06:33.213517 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4764375-e1cb-4e85-9f66-4a087de2a863 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:33.231457 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b4923-e9ac-4c0b-b4ad-d48c210ed1ed None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:33.248647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:33.273737 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79963807-53fb-4bdd-a0d6-7ffc9b4ffc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:33.274905 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79963807-53fb-4bdd-a0d6-7ffc9b4ffc89 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:33.275364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79963807-53fb-4bdd-a0d6-7ffc9b4ffc89 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:33.275860 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79963807-53fb-4bdd-a0d6-7ffc9b4ffc89 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:33.276680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79963807-53fb-4bdd-a0d6-7ffc9b4ffc89 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:33.279082 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79963807-53fb-4bdd-a0d6-7ffc9b4ffc89 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:33.288215 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-79963807-53fb-4bdd-a0d6-7ffc9b4ffc89 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 12:06:33.288215 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:33.288215 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:06:33.288215 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:06:33.288215 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:06:33.288215 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:06:33.288215 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:06:33.288215 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:33.288215 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 12:06:33.288215 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:33.288215 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:06:33.288215 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:33.288215 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:33.288215 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:33.288215 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:33.288215 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:33.288215 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:33.289285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:33.289285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:33.289285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:33.289285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:33.289285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:33.289285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:33.289285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:06:33.289285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:06:33.289285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:06:33.289285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:06:33.289285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:33.289285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:33.289285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:33.289285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:33.289285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:33.289285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:33.290165 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:33.290165 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:33.290165 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:33.290165 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:33.290165 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:33.290165 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:33.290165 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:33.290165 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:33.290165 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:33.290165 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:33.290165 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:06:33.290165 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:33.290165 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 12:06:33.290165 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:33.292571 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 419/852] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Sep 5 12:06:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:33.293453 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79963807-53fb-4bdd-a0d6-7ffc9b4ffc89 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:33.319143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:33.328077 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4764375-e1cb-4e85-9f66-4a087de2a863 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:33.368115 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6180c45-34e6-4b61-9308-efb68885be61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:33.369051 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6180c45-34e6-4b61-9308-efb68885be61 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:33.369539 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6180c45-34e6-4b61-9308-efb68885be61 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:33.370010 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6180c45-34e6-4b61-9308-efb68885be61 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:33.378068 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6180c45-34e6-4b61-9308-efb68885be61 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:33.379507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6180c45-34e6-4b61-9308-efb68885be61 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:33.392321 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-f6180c45-34e6-4b61-9308-efb68885be61 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 12:06:33.392321 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:33.392321 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:33.392321 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:33.392321 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:33.392321 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:33.392321 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:33.392321 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:33.392321 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:33.392321 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:33.392321 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:33.392321 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:33.392321 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:06:33.392321 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:06:33.392321 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:06:33.392321 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:06:33.393347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:33.393347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:33.393347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:06:33.393347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:33.393347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:06:33.393347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:33.393347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:33.393347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:33.393347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:33.393347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:33.393347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:33.393347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:33.393347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:33.393347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:33.393347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:33.393347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:33.393347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:33.393347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:33.394345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:33.394345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:33.394345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:06:33.394345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:06:33.394345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:06:33.394345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:06:33.394345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:06:33.394345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:06:33.394345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 12:06:33.394345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:33.394345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 434/853] 10.222.0.16 () {56 vars in 1117 bytes} [Mon Sep 5 12:06:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:33.394345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6180c45-34e6-4b61-9308-efb68885be61 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:33.415575 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79963807-53fb-4bdd-a0d6-7ffc9b4ffc89 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:33.452962 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554b5bf-2697-4716-ac24-584bb6e6c837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:33.453591 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554b5bf-2697-4716-ac24-584bb6e6c837 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:33.453867 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554b5bf-2697-4716-ac24-584bb6e6c837 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:33.454190 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554b5bf-2697-4716-ac24-584bb6e6c837 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:33.462208 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e554b5bf-2697-4716-ac24-584bb6e6c837 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:33.464901 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e554b5bf-2697-4716-ac24-584bb6e6c837 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:33.479581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-e554b5bf-2697-4716-ac24-584bb6e6c837 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:33.479581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:33.479581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:33.479581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:33.479581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:33.479581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:33.479581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:33.479581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:33.479581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:33.479581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:33.479581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:33.479581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:33.479581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:06:33.479581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:06:33.479581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:06:33.480523 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:06:33.480523 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:33.480523 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:33.480523 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:33.480523 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:33.480523 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:33.480523 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:33.480523 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:33.480523 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:33.480523 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:33.480523 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:33.480523 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:33.480523 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:33.480523 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:33.480523 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:33.480523 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:33.480523 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:33.481488 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:06:33.481488 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:06:33.481488 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:06:33.481488 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:06:33.481488 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:06:33.481488 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:06:33.481488 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:33.481488 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:33.481488 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 420/854] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:06:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:33.481488 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554b5bf-2697-4716-ac24-584bb6e6c837 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:33.496736 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:33.508223 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6180c45-34e6-4b61-9308-efb68885be61 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:33.548999 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8180f-c5d9-4e21-9546-e046d521a9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:33.549746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8180f-c5d9-4e21-9546-e046d521a9a0 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:33.550025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8180f-c5d9-4e21-9546-e046d521a9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:33.550357 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8180f-c5d9-4e21-9546-e046d521a9a0 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:33.550988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a8180f-c5d9-4e21-9546-e046d521a9a0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:33.551313 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a8180f-c5d9-4e21-9546-e046d521a9a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:33.552733 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a8180f-c5d9-4e21-9546-e046d521a9a0 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:33.557440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 435/855] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Sep 5 12:06:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:33.558238 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8180f-c5d9-4e21-9546-e046d521a9a0 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:33.573405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:33.593014 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554b5bf-2697-4716-ac24-584bb6e6c837 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:33.632725 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0e8ea9-dac6-48d4-93c5-68ee8a083703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:33.633465 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e8ea9-dac6-48d4-93c5-68ee8a083703 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:33.633744 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e8ea9-dac6-48d4-93c5-68ee8a083703 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:33.634124 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e8ea9-dac6-48d4-93c5-68ee8a083703 None admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:33.634890 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0e8ea9-dac6-48d4-93c5-68ee8a083703 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:33.635196 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0e8ea9-dac6-48d4-93c5-68ee8a083703 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:33.636681 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0e8ea9-dac6-48d4-93c5-68ee8a083703 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:33.662226 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 421/856] 10.222.0.16 () {56 vars in 1149 bytes} [Mon Sep 5 12:06:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:33.662226 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0e8ea9-dac6-48d4-93c5-68ee8a083703 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:33.685331 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8180f-c5d9-4e21-9546-e046d521a9a0 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:33.722457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547b8e0-24e4-4624-b1ba-ffc5ec71b29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:33.723072 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547b8e0-24e4-4624-b1ba-ffc5ec71b29c None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:33.723354 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547b8e0-24e4-4624-b1ba-ffc5ec71b29c None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:33.723647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547b8e0-24e4-4624-b1ba-ffc5ec71b29c None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:33.724356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0547b8e0-24e4-4624-b1ba-ffc5ec71b29c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:33.724658 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0547b8e0-24e4-4624-b1ba-ffc5ec71b29c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:33.726131 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0547b8e0-24e4-4624-b1ba-ffc5ec71b29c None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:33.739087 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 436/857] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:06:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:33.739995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547b8e0-24e4-4624-b1ba-ffc5ec71b29c None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:33.761034 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:33.788543 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0e8ea9-dac6-48d4-93c5-68ee8a083703 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:33.836005 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3302131-57de-46e6-82e0-a2b13318f045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:33.836702 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3302131-57de-46e6-82e0-a2b13318f045 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:33.836917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3302131-57de-46e6-82e0-a2b13318f045 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:33.837155 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3302131-57de-46e6-82e0-a2b13318f045 None admin] PATH_INFO: `/v3/services` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:33.837788 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3302131-57de-46e6-82e0-a2b13318f045 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:33.838271 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3302131-57de-46e6-82e0-a2b13318f045 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:33.839459 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3302131-57de-46e6-82e0-a2b13318f045 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:33.844854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 422/858] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Sep 5 12:06:33 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:33.855745 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547b8e0-24e4-4624-b1ba-ffc5ec71b29c None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:33.857037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3302131-57de-46e6-82e0-a2b13318f045 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:33.871325 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:33.903938 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16785ff6-04ff-41f3-b47f-a608e18ea57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:33.904600 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16785ff6-04ff-41f3-b47f-a608e18ea57a None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:33.904893 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16785ff6-04ff-41f3-b47f-a608e18ea57a None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:33.905168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16785ff6-04ff-41f3-b47f-a608e18ea57a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:34.007986 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16785ff6-04ff-41f3-b47f-a608e18ea57a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:34.009457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16785ff6-04ff-41f3-b47f-a608e18ea57a None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:34.013566 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3302131-57de-46e6-82e0-a2b13318f045 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:34.026711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:34.054520 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d794d1-6505-4264-83ce-59d37b4f55d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:34.055137 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d794d1-6505-4264-83ce-59d37b4f55d2 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:34.055418 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d794d1-6505-4264-83ce-59d37b4f55d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:34.055750 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d794d1-6505-4264-83ce-59d37b4f55d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:34.056502 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d794d1-6505-4264-83ce-59d37b4f55d2 None admin] RBAC: Adding query filter params (role.id=618c48706f7f445abeddf7eb2f8e927a, scope.project.id=fa3178684d184ec880f17640d19d55b7, user.id=857eb7657e7d45938baca17f91a16901) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:34.056815 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d794d1-6505-4264-83ce-59d37b4f55d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:34.058263 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d794d1-6505-4264-83ce-59d37b4f55d2 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:34.066182 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 423/859] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:06:33 2022] GET /identity/v3/role_assignments?user.id=857eb7657e7d45938baca17f91a16901&scope.project.id=fa3178684d184ec880f17640d19d55b7&role.id=618c48706f7f445abeddf7eb2f8e927a => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:34.067677 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf32bd6-02e8-4f3c-aaf5-e998fd912952 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:34.067807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf32bd6-02e8-4f3c-aaf5-e998fd912952 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:34.068049 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf32bd6-02e8-4f3c-aaf5-e998fd912952 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:34.069399 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 424/860] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:34.074844 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd413501-4ea9-43ee-8282-77978f040757 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:34.075248 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd413501-4ea9-43ee-8282-77978f040757 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:34.075476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd413501-4ea9-43ee-8282-77978f040757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:34.099946 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-bd413501-4ea9-43ee-8282-77978f040757 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:34.101964 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-bd413501-4ea9-43ee-8282-77978f040757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:34.155932 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 437/861] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:06:33 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:06:34.164778 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16785ff6-04ff-41f3-b47f-a608e18ea57a None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:34.189176 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:34.200174 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 425/862] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:34 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:06:34.293052 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16785ff6-04ff-41f3-b47f-a608e18ea57a None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:34.309769 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:34.336784 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b364e272-7b44-4d90-9777-99a35a363b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:34.337422 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364e272-7b44-4d90-9777-99a35a363b73 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:34.337704 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364e272-7b44-4d90-9777-99a35a363b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:34.337984 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364e272-7b44-4d90-9777-99a35a363b73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:34.351245 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b364e272-7b44-4d90-9777-99a35a363b73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:34.352822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b364e272-7b44-4d90-9777-99a35a363b73 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:34.369030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-b364e272-7b44-4d90-9777-99a35a363b73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:34.369030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:34.369030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:34.369030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:34.369030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:34.369030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:34.369030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:34.369030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:34.369030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:34.369030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:34.369030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:34.369030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:34.369030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:34.369030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:34.369030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:34.369030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:34.370073 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:34.370073 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:34.370073 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:34.370073 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:34.370073 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:34.370073 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:34.370073 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:34.370073 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:34.370073 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:34.370073 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:34.370073 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:34.370073 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:34.370073 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:34.370073 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:34.370073 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:34.370073 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:34.370073 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:34.370977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:34.370977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:34.370977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:34.370977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 438/863] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:34.379581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd413501-4ea9-43ee-8282-77978f040757 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:34.484705 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd413501-4ea9-43ee-8282-77978f040757 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:34.491781 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b364e272-7b44-4d90-9777-99a35a363b73 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:34.525707 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4170c6-1a03-4936-b103-6f116dda7418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:34.526543 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4170c6-1a03-4936-b103-6f116dda7418 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:34.527003 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4170c6-1a03-4936-b103-6f116dda7418 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:34.527465 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4170c6-1a03-4936-b103-6f116dda7418 None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:34.529022 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4170c6-1a03-4936-b103-6f116dda7418 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:34.529354 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4170c6-1a03-4936-b103-6f116dda7418 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:34.530800 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4170c6-1a03-4936-b103-6f116dda7418 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:34.545774 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 426/864] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:34.552033 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4170c6-1a03-4936-b103-6f116dda7418 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:34.622900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:34.634680 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b364e272-7b44-4d90-9777-99a35a363b73 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:34.676383 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4170c6-1a03-4936-b103-6f116dda7418 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:34.683550 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f9652-aed3-40d7-ac7b-6734bc4b9008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:34.685120 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f9652-aed3-40d7-ac7b-6734bc4b9008 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:34.685623 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f9652-aed3-40d7-ac7b-6734bc4b9008 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:34.686124 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f9652-aed3-40d7-ac7b-6734bc4b9008 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:34.687397 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705f9652-aed3-40d7-ac7b-6734bc4b9008 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:34.689748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705f9652-aed3-40d7-ac7b-6734bc4b9008 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:34.703224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-705f9652-aed3-40d7-ac7b-6734bc4b9008 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 12:06:34.703224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:34.703224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:06:34.703224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:06:34.703224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:06:34.703224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:06:34.703224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:06:34.703224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:34.703224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 12:06:34.703224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:34.703224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:06:34.703224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:34.703224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:34.703224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:34.703224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:34.703224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:34.704299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:34.704299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:34.704299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:34.704299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:34.704299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:34.704299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:34.704299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:34.704299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:06:34.704299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:06:34.704299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:06:34.704299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:06:34.704299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:34.704299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:34.704299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:34.704299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:34.704299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:34.704299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:34.705211 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:34.705211 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:34.705211 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:34.705211 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:34.705211 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:34.705211 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:34.705211 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:34.705211 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:34.705211 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:34.705211 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:34.705211 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:06:34.705211 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:34.705211 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 12:06:34.705211 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:34.711153 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 439/865] 10.222.0.16 () {56 vars in 1132 bytes} [Mon Sep 5 12:06:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:34.713551 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f9652-aed3-40d7-ac7b-6734bc4b9008 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:34.723234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445e96b-40af-4852-af27-a13c018f51e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:34.723902 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445e96b-40af-4852-af27-a13c018f51e9 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:34.724178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445e96b-40af-4852-af27-a13c018f51e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:34.724446 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445e96b-40af-4852-af27-a13c018f51e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:34.732157 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445e96b-40af-4852-af27-a13c018f51e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:34.733597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445e96b-40af-4852-af27-a13c018f51e9 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:34.748749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-7445e96b-40af-4852-af27-a13c018f51e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:34.748749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:34.748749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:34.748749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:34.748749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:34.748749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:34.748749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:34.748749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:34.748749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:34.748749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:34.748749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:34.748749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:34.748749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:06:34.748749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:06:34.748749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:06:34.749762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:06:34.749762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:34.749762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:34.749762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:34.749762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:34.749762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:34.749762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:34.749762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:34.749762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:34.749762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:34.749762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:34.749762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:34.749762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:34.749762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:34.749762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:34.749762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:34.749762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:34.750924 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:06:34.750924 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:06:34.750924 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:06:34.750924 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:06:34.750924 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:06:34.750924 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:06:34.750924 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:34.750924 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:34.750924 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 427/866] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:06:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:06:34.754369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:34.761926 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445e96b-40af-4852-af27-a13c018f51e9 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:34.837700 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f9652-aed3-40d7-ac7b-6734bc4b9008 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:34.876422 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6c745-565a-48ee-b0b2-8873d31fb5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:34.877032 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6c745-565a-48ee-b0b2-8873d31fb5e3 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:34.877308 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6c745-565a-48ee-b0b2-8873d31fb5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:34.877574 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6c745-565a-48ee-b0b2-8873d31fb5e3 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:34.878198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd6c745-565a-48ee-b0b2-8873d31fb5e3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:34.878497 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd6c745-565a-48ee-b0b2-8873d31fb5e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:34.879913 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd6c745-565a-48ee-b0b2-8873d31fb5e3 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:34.885046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 440/867] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Sep 5 12:06:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:06:34.891546 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6c745-565a-48ee-b0b2-8873d31fb5e3 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:34.896203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445e96b-40af-4852-af27-a13c018f51e9 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:34.925400 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:34.941162 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85672b3c-31ac-44bf-b8b4-42b36b4a402e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:34.941770 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85672b3c-31ac-44bf-b8b4-42b36b4a402e None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:34.942048 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85672b3c-31ac-44bf-b8b4-42b36b4a402e None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:34.942320 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85672b3c-31ac-44bf-b8b4-42b36b4a402e None admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:34.943014 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85672b3c-31ac-44bf-b8b4-42b36b4a402e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:34.943315 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85672b3c-31ac-44bf-b8b4-42b36b4a402e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:34.945027 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85672b3c-31ac-44bf-b8b4-42b36b4a402e None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:34.959540 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 428/868] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:06:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:34.965432 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85672b3c-31ac-44bf-b8b4-42b36b4a402e None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:35.015513 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:35.025349 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6c745-565a-48ee-b0b2-8873d31fb5e3 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:35.076750 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe65b2-e81e-4a1f-a335-04216cb99a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:35.078079 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe65b2-e81e-4a1f-a335-04216cb99a7f None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:35.078531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe65b2-e81e-4a1f-a335-04216cb99a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:35.078948 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe65b2-e81e-4a1f-a335-04216cb99a7f None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:35.079760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbe65b2-e81e-4a1f-a335-04216cb99a7f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:35.080217 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbe65b2-e81e-4a1f-a335-04216cb99a7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:35.082500 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbe65b2-e81e-4a1f-a335-04216cb99a7f None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:35.089111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 441/869] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Sep 5 12:06:34 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:06:35.104133 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85672b3c-31ac-44bf-b8b4-42b36b4a402e None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:35.146060 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb140634-a85b-4a62-9a81-efdbb8e18f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:35.146868 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb140634-a85b-4a62-9a81-efdbb8e18f1e None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:35.147323 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb140634-a85b-4a62-9a81-efdbb8e18f1e None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:35.147725 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb140634-a85b-4a62-9a81-efdbb8e18f1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:35.148507 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb140634-a85b-4a62-9a81-efdbb8e18f1e None admin] RBAC: Adding query filter params (role.id=8cf764a0144a490a822fcf86d7aceeb0, scope.project.id=fa3178684d184ec880f17640d19d55b7, user.id=8c5ae98336c441118d01f46a50f732d7) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:35.148821 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb140634-a85b-4a62-9a81-efdbb8e18f1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:35.150300 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb140634-a85b-4a62-9a81-efdbb8e18f1e None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:35.157898 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 429/870] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:06:34 2022] GET /identity/v3/role_assignments?user.id=8c5ae98336c441118d01f46a50f732d7&scope.project.id=fa3178684d184ec880f17640d19d55b7&role.id=8cf764a0144a490a822fcf86d7aceeb0 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:06:35.874790 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21703b8-628f-4cbe-8835-c7cd57228b0e None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:35.875849 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21703b8-628f-4cbe-8835-c7cd57228b0e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:35.876334 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21703b8-628f-4cbe-8835-c7cd57228b0e None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:35.878159 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 442/871] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:35.883733 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac6102-8a08-4dbd-acb1-3184552190ac None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:35.884593 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac6102-8a08-4dbd-acb1-3184552190ac None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:35.885059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac6102-8a08-4dbd-acb1-3184552190ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:35.905080 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-78ac6102-8a08-4dbd-acb1-3184552190ac None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:35.906606 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-78ac6102-8a08-4dbd-acb1-3184552190ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:35.948754 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:35.978571 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 430/872] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:35 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:06:36.188700 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062554e-a24b-4658-8174-27bbc916ebe0 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:36.188895 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062554e-a24b-4658-8174-27bbc916ebe0 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:36.189162 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062554e-a24b-4658-8174-27bbc916ebe0 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:36.190786 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 443/873] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:36.196545 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088b39a-fd00-4b1a-a61d-1c11fd57f9cd None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:36.196836 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088b39a-fd00-4b1a-a61d-1c11fd57f9cd None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:36.197117 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088b39a-fd00-4b1a-a61d-1c11fd57f9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:36.218107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-1088b39a-fd00-4b1a-a61d-1c11fd57f9cd None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:36.219359 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-1088b39a-fd00-4b1a-a61d-1c11fd57f9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:36.253818 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062554e-a24b-4658-8174-27bbc916ebe0 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:36.294880 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 431/874] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:06:36.364033 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062554e-a24b-4658-8174-27bbc916ebe0 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:36.400369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bbe6f8-9bfb-43c7-a5b6-d2d3e2802dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:36.401026 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbe6f8-9bfb-43c7-a5b6-d2d3e2802dae None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:36.401341 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbe6f8-9bfb-43c7-a5b6-d2d3e2802dae None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:36.401732 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbe6f8-9bfb-43c7-a5b6-d2d3e2802dae None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:36.403066 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bbe6f8-9bfb-43c7-a5b6-d2d3e2802dae None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:36.404554 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bbe6f8-9bfb-43c7-a5b6-d2d3e2802dae None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:36.422136 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 444/875] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Sep 5 12:06:36 2022] POST /identity/v3/services => generated 251 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:06:36.578725 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1088b39a-fd00-4b1a-a61d-1c11fd57f9cd None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:36.695625 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1088b39a-fd00-4b1a-a61d-1c11fd57f9cd None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:36.698112 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:36.744985 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6aab63-a3bd-4dfd-9cbb-bfc1b3231234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:36.745634 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6aab63-a3bd-4dfd-9cbb-bfc1b3231234 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:36.745951 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6aab63-a3bd-4dfd-9cbb-bfc1b3231234 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:36.746227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6aab63-a3bd-4dfd-9cbb-bfc1b3231234 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:36.746853 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6aab63-a3bd-4dfd-9cbb-bfc1b3231234 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:36.748401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6aab63-a3bd-4dfd-9cbb-bfc1b3231234 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:36.758857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-dc6aab63-a3bd-4dfd-9cbb-bfc1b3231234 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 12:06:36.758857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:36.758857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:06:36.758857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:06:36.758857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:06:36.758857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:06:36.758857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:06:36.758857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:36.758857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 12:06:36.758857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:36.758857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:06:36.758857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:36.758857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:36.758857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:36.758857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:36.758857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:36.758857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:36.759952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:36.759952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:36.759952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:36.759952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:36.759952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:36.759952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:36.759952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:06:36.759952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:06:36.759952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:06:36.759952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:06:36.759952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:36.759952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:36.759952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:36.759952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:36.759952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:36.759952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:36.760819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:36.760819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:36.760819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:36.760819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:36.760819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:36.760819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:36.760819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:36.760819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:36.760819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:36.760819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:36.760819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:06:36.760819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:36.760819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 12:06:36.760819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:36.760819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 432/876] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Sep 5 12:06:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:36.766618 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bbe6f8-9bfb-43c7-a5b6-d2d3e2802dae None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:36.793094 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:36.893657 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bbe6f8-9bfb-43c7-a5b6-d2d3e2802dae None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:36.935010 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b4e2fd-891b-433b-b5f8-479348e55e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:36.935650 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4e2fd-891b-433b-b5f8-479348e55e5e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:36.935994 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4e2fd-891b-433b-b5f8-479348e55e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:36.936268 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4e2fd-891b-433b-b5f8-479348e55e5e None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:36.936898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b4e2fd-891b-433b-b5f8-479348e55e5e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:36.937198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b4e2fd-891b-433b-b5f8-479348e55e5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:36.938583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b4e2fd-891b-433b-b5f8-479348e55e5e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:36.943908 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 445/877] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Sep 5 12:06:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:36.952966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6aab63-a3bd-4dfd-9cbb-bfc1b3231234 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:37.066828 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6aab63-a3bd-4dfd-9cbb-bfc1b3231234 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:37.089719 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0aba1-e4b9-4411-bd72-2803b0be16d5 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:37.090892 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0aba1-e4b9-4411-bd72-2803b0be16d5 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:37.091378 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0aba1-e4b9-4411-bd72-2803b0be16d5 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:37.096968 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 446/878] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:06:37.098816 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78b323-ba9e-4b5b-a19d-668cb9aabda1 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:37.099334 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78b323-ba9e-4b5b-a19d-668cb9aabda1 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:37.099837 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78b323-ba9e-4b5b-a19d-668cb9aabda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:37.106807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a723579-d347-4932-8238-2ac62e342c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:37.107417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a723579-d347-4932-8238-2ac62e342c91 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:37.107733 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a723579-d347-4932-8238-2ac62e342c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:37.108004 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a723579-d347-4932-8238-2ac62e342c91 None admin] PATH_INFO: `/v3/services` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:37.108636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a723579-d347-4932-8238-2ac62e342c91 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:37.109122 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a723579-d347-4932-8238-2ac62e342c91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:37.111370 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a723579-d347-4932-8238-2ac62e342c91 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:37.117296 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 433/879] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Sep 5 12:06:36 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:37.131558 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-3a78b323-ba9e-4b5b-a19d-668cb9aabda1 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:37.131558 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3a78b323-ba9e-4b5b-a19d-668cb9aabda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:37.207158 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 447/880] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:37 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:06:37.226830 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be690dc-7a99-4557-a243-c8300ed2bd3f None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:37.227454 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be690dc-7a99-4557-a243-c8300ed2bd3f None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:37.227951 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be690dc-7a99-4557-a243-c8300ed2bd3f None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:37.229840 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 434/881] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:37.236335 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6f82d-6b14-44c2-98d7-7d1f282ec80b None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:37.236908 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6f82d-6b14-44c2-98d7-7d1f282ec80b None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:37.237385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6f82d-6b14-44c2-98d7-7d1f282ec80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:37.260372 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-c3a6f82d-6b14-44c2-98d7-7d1f282ec80b None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:37.261861 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c3a6f82d-6b14-44c2-98d7-7d1f282ec80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:37.270271 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:37.334908 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 448/882] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:37 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:06:37.484950 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be690dc-7a99-4557-a243-c8300ed2bd3f None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:37.514318 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:37.596687 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be690dc-7a99-4557-a243-c8300ed2bd3f None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:37.620016 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6f82d-6b14-44c2-98d7-7d1f282ec80b None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:37.633745 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7c975-dad2-4661-9bea-5a8255b59572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:37.634645 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7c975-dad2-4661-9bea-5a8255b59572 None admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:37.635070 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7c975-dad2-4661-9bea-5a8255b59572 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:37.635486 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7c975-dad2-4661-9bea-5a8255b59572 None admin] PATH_INFO: `/v3/services` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:37.636330 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e7c975-dad2-4661-9bea-5a8255b59572 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:37.638415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e7c975-dad2-4661-9bea-5a8255b59572 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:37.664816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 435/883] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Sep 5 12:06:37 2022] POST /identity/v3/services => generated 269 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:06:37.755833 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6f82d-6b14-44c2-98d7-7d1f282ec80b None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:37.801976 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dcb808-ca1d-44ec-9137-618699985221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:37.802684 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dcb808-ca1d-44ec-9137-618699985221 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:37.802967 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dcb808-ca1d-44ec-9137-618699985221 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:37.803236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dcb808-ca1d-44ec-9137-618699985221 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:37.811552 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:37.815716 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dcb808-ca1d-44ec-9137-618699985221 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:06:37.821869 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-63dcb808-ca1d-44ec-9137-618699985221 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 12:06:37.821869 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:37.821869 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:37.821869 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:37.821869 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:37.821869 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:37.821869 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:37.821869 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:37.821869 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:37.821869 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:37.821869 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:37.821869 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:37.821869 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:06:37.821869 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:06:37.821869 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:06:37.821869 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:06:37.822859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:06:37.822859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:06:37.822859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:06:37.822859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:06:37.822859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:37.822859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:37.822859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:37.822859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:37.822859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:37.822859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:37.822859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:37.822859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:37.822859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:37.822859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:37.822859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:37.822859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:37.823737 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:37.823737 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:37.823737 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:37.823737 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:37.823737 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:06:37.823737 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:06:37.823737 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:06:37.823737 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:06:37.823737 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:06:37.823737 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:06:37.823737 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 12:06:37.823737 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:37.823737 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 449/884] 10.222.0.16 () {56 vars in 1120 bytes} [Mon Sep 5 12:06:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:06:37.832302 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7c975-dad2-4661-9bea-5a8255b59572 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:37.930180 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:37.944702 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7c975-dad2-4661-9bea-5a8255b59572 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:37.979407 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01682e0b-6bb6-457f-9d20-121c2bf47233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:37.980064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01682e0b-6bb6-457f-9d20-121c2bf47233 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:37.980313 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01682e0b-6bb6-457f-9d20-121c2bf47233 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:37.980589 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01682e0b-6bb6-457f-9d20-121c2bf47233 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:37.981782 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01682e0b-6bb6-457f-9d20-121c2bf47233 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:37.982090 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01682e0b-6bb6-457f-9d20-121c2bf47233 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:37.983448 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01682e0b-6bb6-457f-9d20-121c2bf47233 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:37.992381 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 436/885] 10.222.0.16 () {56 vars in 1115 bytes} [Mon Sep 5 12:06:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:37.999648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dcb808-ca1d-44ec-9137-618699985221 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:38.104157 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dcb808-ca1d-44ec-9137-618699985221 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:38.140169 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250d330-85ad-4e8a-ac08-673b64742605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:38.141240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250d330-85ad-4e8a-ac08-673b64742605 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:38.141748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250d330-85ad-4e8a-ac08-673b64742605 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:38.142304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250d330-85ad-4e8a-ac08-673b64742605 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:38.154233 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2250d330-85ad-4e8a-ac08-673b64742605 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:38.156031 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2250d330-85ad-4e8a-ac08-673b64742605 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:38.175381 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-2250d330-85ad-4e8a-ac08-673b64742605 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:38.175381 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:38.175381 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:38.175381 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:38.175381 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:38.175381 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:38.175381 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:38.175381 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:38.175381 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:38.175381 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:38.175381 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:38.175381 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:38.175381 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:38.175381 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:38.175381 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:38.175381 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:38.176507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:38.176507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:38.176507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:38.176507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:38.176507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:38.176507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:38.176507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:38.176507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:38.176507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:38.176507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:38.176507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:38.176507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:38.176507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:38.176507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:38.176507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:38.176507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:38.176507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:38.177594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:38.177594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:38.177594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:38.177594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 450/886] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:38.184170 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01682e0b-6bb6-457f-9d20-121c2bf47233 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:38.298973 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01682e0b-6bb6-457f-9d20-121c2bf47233 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:38.338624 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9af391-391b-4716-8be0-53fd4777511d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:38.339624 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9af391-391b-4716-8be0-53fd4777511d None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:38.340107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9af391-391b-4716-8be0-53fd4777511d None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:38.341127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9af391-391b-4716-8be0-53fd4777511d None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:38.341990 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9af391-391b-4716-8be0-53fd4777511d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:38.342491 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9af391-391b-4716-8be0-53fd4777511d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:38.344084 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9af391-391b-4716-8be0-53fd4777511d None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:38.357634 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 437/887] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:38.364592 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250d330-85ad-4e8a-ac08-673b64742605 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:38.424906 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:38.478288 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250d330-85ad-4e8a-ac08-673b64742605 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:38.513765 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d63e5e2-6663-4526-a5dd-f59711565221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:38.514775 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63e5e2-6663-4526-a5dd-f59711565221 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:38.515254 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63e5e2-6663-4526-a5dd-f59711565221 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:38.515841 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63e5e2-6663-4526-a5dd-f59711565221 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:38.523978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d63e5e2-6663-4526-a5dd-f59711565221 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:38.525701 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d63e5e2-6663-4526-a5dd-f59711565221 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:38.538444 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-2d63e5e2-6663-4526-a5dd-f59711565221 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 12:06:38.538444 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:38.538444 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:38.538444 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:38.538444 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:38.538444 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:38.538444 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:38.538444 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:38.538444 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:38.538444 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:38.538444 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:38.538444 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:38.538444 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:06:38.538444 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:06:38.538444 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:06:38.538444 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:06:38.539460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:38.539460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:38.539460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:06:38.539460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:38.539460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:06:38.539460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:38.539460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:38.539460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:38.539460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:38.539460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:38.539460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:38.539460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:38.539460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:38.539460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:38.539460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:38.539460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:38.539460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:38.539460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:38.540459 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:38.540459 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:38.540459 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:06:38.540459 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:06:38.540459 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:06:38.540459 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:06:38.540459 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:06:38.540459 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:06:38.540459 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 12:06:38.540459 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:38.542993 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 451/888] 10.222.0.16 () {56 vars in 1117 bytes} [Mon Sep 5 12:06:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:38.549006 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9af391-391b-4716-8be0-53fd4777511d None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:38.562529 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:38.563997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2ba36-6de8-4735-aa56-40e878cc7d1c None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:38.564282 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2ba36-6de8-4735-aa56-40e878cc7d1c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:38.564558 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2ba36-6de8-4735-aa56-40e878cc7d1c None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:38.566493 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 452/889] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:38.573740 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2871e09-eb5f-4630-95a1-179385889288 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:38.574024 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2871e09-eb5f-4630-95a1-179385889288 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:38.574294 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2871e09-eb5f-4630-95a1-179385889288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:38.594970 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-a2871e09-eb5f-4630-95a1-179385889288 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:38.596764 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a2871e09-eb5f-4630-95a1-179385889288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:38.669170 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9af391-391b-4716-8be0-53fd4777511d None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:38.691560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 453/890] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:38 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:06:38.713615 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af036413-f484-4e85-a861-a9f620d0e7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:38.714271 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af036413-f484-4e85-a861-a9f620d0e7e7 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:38.714555 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af036413-f484-4e85-a861-a9f620d0e7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:38.714830 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af036413-f484-4e85-a861-a9f620d0e7e7 None admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:38.716171 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af036413-f484-4e85-a861-a9f620d0e7e7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:38.716515 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af036413-f484-4e85-a861-a9f620d0e7e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:38.717932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af036413-f484-4e85-a861-a9f620d0e7e7 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:38.736587 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 438/891] 10.222.0.16 () {56 vars in 1149 bytes} [Mon Sep 5 12:06:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:38.742285 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2871e09-eb5f-4630-95a1-179385889288 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:38.787640 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:38.852967 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2871e09-eb5f-4630-95a1-179385889288 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:38.889853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549bf65a-a7ac-4fab-b2b5-406dd2d54f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:38.890852 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549bf65a-a7ac-4fab-b2b5-406dd2d54f4d None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:38.891346 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549bf65a-a7ac-4fab-b2b5-406dd2d54f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:38.891845 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549bf65a-a7ac-4fab-b2b5-406dd2d54f4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:38.971463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af036413-f484-4e85-a861-a9f620d0e7e7 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:38.975459 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549bf65a-a7ac-4fab-b2b5-406dd2d54f4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:38.976990 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549bf65a-a7ac-4fab-b2b5-406dd2d54f4d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:39.023930 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:39.044365 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:39.086259 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af036413-f484-4e85-a861-a9f620d0e7e7 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:39.104228 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 454/892] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:06:38 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:06:39.109913 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549bf65a-a7ac-4fab-b2b5-406dd2d54f4d None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:39.126696 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e52ec-54a7-4464-9a80-ba94e3fba7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:39.127776 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e52ec-54a7-4464-9a80-ba94e3fba7ba None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:39.128160 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e52ec-54a7-4464-9a80-ba94e3fba7ba None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:39.128534 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e52ec-54a7-4464-9a80-ba94e3fba7ba None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:39.129381 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e52ec-54a7-4464-9a80-ba94e3fba7ba None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:39.131146 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e52ec-54a7-4464-9a80-ba94e3fba7ba None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:39.143813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-e64e52ec-54a7-4464-9a80-ba94e3fba7ba None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 12:06:39.143813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:39.143813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:06:39.143813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:06:39.143813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:06:39.143813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:06:39.143813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:06:39.143813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:39.143813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 12:06:39.143813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:39.143813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:06:39.143813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:39.143813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:39.143813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:39.143813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:39.143813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:39.143813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:39.144968 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:39.144968 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:39.144968 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:39.144968 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:39.144968 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:39.144968 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:39.144968 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:06:39.144968 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:06:39.144968 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:06:39.144968 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:06:39.144968 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:39.144968 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:39.144968 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:39.144968 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:39.144968 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:39.144968 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:39.145843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:39.145843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:39.145843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:39.145843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:39.145843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:39.145843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:39.145843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:39.145843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:39.145843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:39.145843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:39.145843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:06:39.145843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:39.145843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 12:06:39.145843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:39.145843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 439/893] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Sep 5 12:06:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:39.151771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e52ec-54a7-4464-9a80-ba94e3fba7ba None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:39.220491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549bf65a-a7ac-4fab-b2b5-406dd2d54f4d None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:39.256402 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cededc96-f976-46cf-af4f-a4020441effe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:39.257251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cededc96-f976-46cf-af4f-a4020441effe None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:39.257690 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cededc96-f976-46cf-af4f-a4020441effe None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:39.257959 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cededc96-f976-46cf-af4f-a4020441effe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:39.263119 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e52ec-54a7-4464-9a80-ba94e3fba7ba None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:39.267657 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:39.269982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cededc96-f976-46cf-af4f-a4020441effe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:39.271909 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cededc96-f976-46cf-af4f-a4020441effe None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:39.288466 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-cededc96-f976-46cf-af4f-a4020441effe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:39.288466 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:39.288466 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:39.288466 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:39.288466 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:39.288466 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:39.288466 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:39.288466 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:39.288466 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:39.288466 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:39.288466 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:39.288466 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:39.288466 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:39.288466 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:39.288466 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:39.288466 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:39.289491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:39.289491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:39.289491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:39.289491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:39.289491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:39.289491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:39.289491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:39.289491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:39.289491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:39.289491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:39.289491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:39.289491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:39.289491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:39.289491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:39.289491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:39.289491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:39.289491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:39.290431 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:39.290431 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:39.290431 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:39.290431 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 455/894] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:39.291185 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17f523-0554-453d-b0be-104e03601055 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:39.291425 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17f523-0554-453d-b0be-104e03601055 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:39.291724 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17f523-0554-453d-b0be-104e03601055 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:39.293125 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 456/895] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:39.298170 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17f523-0554-453d-b0be-104e03601055 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:39.300471 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:39.306897 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0971080a-1f3a-40ab-a937-4da28a39f764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:39.307503 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971080a-1f3a-40ab-a937-4da28a39f764 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:39.307807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971080a-1f3a-40ab-a937-4da28a39f764 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:39.308080 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971080a-1f3a-40ab-a937-4da28a39f764 None admin] PATH_INFO: `/v3/services` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:39.308711 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971080a-1f3a-40ab-a937-4da28a39f764 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:39.309010 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971080a-1f3a-40ab-a937-4da28a39f764 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:39.310985 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971080a-1f3a-40ab-a937-4da28a39f764 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:39.321444 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 440/896] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Sep 5 12:06:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:39.321444 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea9509-e09e-4d31-8b85-5e12b3ef8b4f None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:39.321444 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea9509-e09e-4d31-8b85-5e12b3ef8b4f None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:39.321444 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea9509-e09e-4d31-8b85-5e12b3ef8b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:39.343129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-7bea9509-e09e-4d31-8b85-5e12b3ef8b4f None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:39.344620 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-7bea9509-e09e-4d31-8b85-5e12b3ef8b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:39.405691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17f523-0554-453d-b0be-104e03601055 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:39.427731 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 441/897] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:39 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:06:39.427731 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea9509-e09e-4d31-8b85-5e12b3ef8b4f None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:39.461830 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf410352-5f4d-406d-834a-8f6de324868e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:39.462949 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf410352-5f4d-406d-834a-8f6de324868e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:39.464296 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf410352-5f4d-406d-834a-8f6de324868e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:39.464719 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf410352-5f4d-406d-834a-8f6de324868e None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:39.465576 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf410352-5f4d-406d-834a-8f6de324868e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:39.466039 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf410352-5f4d-406d-834a-8f6de324868e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:39.467566 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf410352-5f4d-406d-834a-8f6de324868e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:39.488338 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 457/898] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:39.489389 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf410352-5f4d-406d-834a-8f6de324868e None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:39.551155 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea9509-e09e-4d31-8b85-5e12b3ef8b4f None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:39.561658 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:39.586297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:39.599266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5ff2d-a8bd-4231-9c7c-18b32e2734fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:39.600163 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5ff2d-a8bd-4231-9c7c-18b32e2734fb None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:39.600619 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5ff2d-a8bd-4231-9c7c-18b32e2734fb None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:39.601094 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5ff2d-a8bd-4231-9c7c-18b32e2734fb None admin] PATH_INFO: `/v3/services` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:39.602401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b5ff2d-a8bd-4231-9c7c-18b32e2734fb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:39.602885 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b5ff2d-a8bd-4231-9c7c-18b32e2734fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:39.604782 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b5ff2d-a8bd-4231-9c7c-18b32e2734fb None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:39.612682 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 442/899] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Sep 5 12:06:39 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:39.618055 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5ff2d-a8bd-4231-9c7c-18b32e2734fb None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:39.619680 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf410352-5f4d-406d-834a-8f6de324868e None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:39.663488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb8fc9-71ae-4aed-af5a-2ef8e81b091b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:39.664448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb8fc9-71ae-4aed-af5a-2ef8e81b091b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:39.665253 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb8fc9-71ae-4aed-af5a-2ef8e81b091b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:39.665733 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb8fc9-71ae-4aed-af5a-2ef8e81b091b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:39.675661 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb8fc9-71ae-4aed-af5a-2ef8e81b091b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:39.677798 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb8fc9-71ae-4aed-af5a-2ef8e81b091b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:39.692545 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-21fb8fc9-71ae-4aed-af5a-2ef8e81b091b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:39.692545 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:39.692545 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:39.692545 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:39.692545 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:39.692545 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:39.692545 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:39.692545 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:39.692545 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:39.692545 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:39.692545 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:39.692545 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:39.692545 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:06:39.692545 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:06:39.692545 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:06:39.693508 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:06:39.693508 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:39.693508 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:39.693508 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:39.693508 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:39.693508 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:39.693508 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:39.693508 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:39.693508 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:39.693508 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:39.693508 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:39.693508 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:39.693508 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:39.693508 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:39.693508 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:39.693508 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:39.693508 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:39.694400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:06:39.694400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:06:39.694400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:06:39.694400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:06:39.694400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:06:39.694400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:06:39.694400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:39.694400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:39.701319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 458/900] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:06:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:39.701319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb8fc9-71ae-4aed-af5a-2ef8e81b091b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:39.736007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5ff2d-a8bd-4231-9c7c-18b32e2734fb None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:39.771812 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c06427c-c103-42a0-bb2b-2b3b00fb5340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:39.772419 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06427c-c103-42a0-bb2b-2b3b00fb5340 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:39.772697 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06427c-c103-42a0-bb2b-2b3b00fb5340 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:39.772967 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06427c-c103-42a0-bb2b-2b3b00fb5340 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:39.773666 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c06427c-c103-42a0-bb2b-2b3b00fb5340 None admin] RBAC: Adding query filter params (interface=public, service_id=ffbd834e3c3e4792958c251446c42d9a, region_id=RegionOne) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:39.773972 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c06427c-c103-42a0-bb2b-2b3b00fb5340 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:39.775315 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c06427c-c103-42a0-bb2b-2b3b00fb5340 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:39.789207 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 443/901] 10.222.0.16 () {56 vars in 1251 bytes} [Mon Sep 5 12:06:39 2022] GET /identity/v3/endpoints?service_id=ffbd834e3c3e4792958c251446c42d9a&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:39.789993 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c06427c-c103-42a0-bb2b-2b3b00fb5340 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:39.808784 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb8fc9-71ae-4aed-af5a-2ef8e81b091b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:39.821514 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:39.843047 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f34c0-2b70-494b-b3bd-75e70dc6e917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:39.843697 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f34c0-2b70-494b-b3bd-75e70dc6e917 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:39.843975 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f34c0-2b70-494b-b3bd-75e70dc6e917 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:39.844244 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f34c0-2b70-494b-b3bd-75e70dc6e917 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:39.844890 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52f34c0-2b70-494b-b3bd-75e70dc6e917 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:39.846251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52f34c0-2b70-494b-b3bd-75e70dc6e917 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:39.860910 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 459/902] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Sep 5 12:06:39 2022] POST /identity/v3/services => generated 241 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:06:39.864612 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f34c0-2b70-494b-b3bd-75e70dc6e917 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:39.882346 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:39.949620 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c06427c-c103-42a0-bb2b-2b3b00fb5340 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:39.989790 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f34c0-2b70-494b-b3bd-75e70dc6e917 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:39.994257 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdce989-e407-4564-b9f9-d3c75d02b4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:39.995055 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdce989-e407-4564-b9f9-d3c75d02b4f4 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:39.995815 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdce989-e407-4564-b9f9-d3c75d02b4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:39.996092 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdce989-e407-4564-b9f9-d3c75d02b4f4 None admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:39.996835 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdce989-e407-4564-b9f9-d3c75d02b4f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:39.997140 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdce989-e407-4564-b9f9-d3c75d02b4f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:39.998577 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdce989-e407-4564-b9f9-d3c75d02b4f4 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:40.011792 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 444/903] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:06:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:40.014131 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd454aeb-ae39-4028-ac31-c83ed69a6159 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:40.014598 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd454aeb-ae39-4028-ac31-c83ed69a6159 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:40.014872 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd454aeb-ae39-4028-ac31-c83ed69a6159 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:40.016330 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 445/904] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:40.020604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd454aeb-ae39-4028-ac31-c83ed69a6159 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:40.033168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7694c97-3948-4a86-8366-8ed28731c783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:40.033800 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7694c97-3948-4a86-8366-8ed28731c783 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:40.034062 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7694c97-3948-4a86-8366-8ed28731c783 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:40.034333 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7694c97-3948-4a86-8366-8ed28731c783 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:40.034945 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7694c97-3948-4a86-8366-8ed28731c783 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:40.036367 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7694c97-3948-4a86-8366-8ed28731c783 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:40.043856 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 460/905] 10.222.0.16 () {56 vars in 1087 bytes} [Mon Sep 5 12:06:39 2022] GET /identity/v3/services => generated 1497 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:06:40.045198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a661e8c-3a76-47cc-a98f-2986d9b52d37 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:40.045491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a661e8c-3a76-47cc-a98f-2986d9b52d37 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:40.045767 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a661e8c-3a76-47cc-a98f-2986d9b52d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:40.068941 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-8a661e8c-3a76-47cc-a98f-2986d9b52d37 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:40.070107 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8a661e8c-3a76-47cc-a98f-2986d9b52d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:40.105333 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:40.179554 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 461/906] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:06:40.181628 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd454aeb-ae39-4028-ac31-c83ed69a6159 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:40.206338 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:40.222636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e50901-d3e2-4413-b053-50f90c558911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:40.223241 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e50901-d3e2-4413-b053-50f90c558911 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:40.223615 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e50901-d3e2-4413-b053-50f90c558911 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:40.223814 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e50901-d3e2-4413-b053-50f90c558911 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:40.224616 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e50901-d3e2-4413-b053-50f90c558911 None admin] RBAC: Adding query filter params (role.id=e4660ff7081a4aaa8e0d840b2e6c6dcd, scope.project.id=fa3178684d184ec880f17640d19d55b7, user.id=8c5ae98336c441118d01f46a50f732d7) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:40.224933 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e50901-d3e2-4413-b053-50f90c558911 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:40.226355 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e50901-d3e2-4413-b053-50f90c558911 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:40.232634 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 446/907] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:06:40 2022] GET /identity/v3/role_assignments?user.id=8c5ae98336c441118d01f46a50f732d7&scope.project.id=fa3178684d184ec880f17640d19d55b7&role.id=e4660ff7081a4aaa8e0d840b2e6c6dcd => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:40.454288 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a661e8c-3a76-47cc-a98f-2986d9b52d37 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:40.566754 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a661e8c-3a76-47cc-a98f-2986d9b52d37 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:40.603078 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ccdf5-b4f5-4f55-a169-582a2b64ac48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:40.603744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ccdf5-b4f5-4f55-a169-582a2b64ac48 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:40.604021 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ccdf5-b4f5-4f55-a169-582a2b64ac48 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:40.604295 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ccdf5-b4f5-4f55-a169-582a2b64ac48 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:40.604878 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ccdf5-b4f5-4f55-a169-582a2b64ac48 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:40.606273 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ccdf5-b4f5-4f55-a169-582a2b64ac48 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:40.615240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-368ccdf5-b4f5-4f55-a169-582a2b64ac48 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 12:06:40.615240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:40.615240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:06:40.615240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:06:40.615240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:06:40.615240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:06:40.615240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:06:40.615240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:40.615240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 12:06:40.615240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:40.615240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:06:40.615240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:40.615240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:40.615240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:40.615240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:40.615240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:40.616278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:40.616278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:40.616278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:40.616278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:40.616278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:40.616278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:40.616278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:40.616278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:06:40.616278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:06:40.616278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:06:40.616278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:06:40.616278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:40.616278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:40.616278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:40.616278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:40.616278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:40.616278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:40.617191 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:40.617191 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:40.617191 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:40.617191 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:40.617191 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:40.617191 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:40.617191 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:40.617191 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:40.617191 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:40.617191 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:40.617191 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:06:40.617191 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:40.617191 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 12:06:40.617191 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:40.617191 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 462/908] 10.222.0.16 () {56 vars in 1132 bytes} [Mon Sep 5 12:06:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:40.622779 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e50901-d3e2-4413-b053-50f90c558911 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:40.735614 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e50901-d3e2-4413-b053-50f90c558911 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:40.771338 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b945e49e-c1ea-4fd7-93b5-5796a42acaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:40.772354 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945e49e-c1ea-4fd7-93b5-5796a42acaa5 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:40.772839 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945e49e-c1ea-4fd7-93b5-5796a42acaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:40.773313 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945e49e-c1ea-4fd7-93b5-5796a42acaa5 None admin] PATH_INFO: `/v3/services` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:40.774164 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b945e49e-c1ea-4fd7-93b5-5796a42acaa5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:40.774669 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b945e49e-c1ea-4fd7-93b5-5796a42acaa5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:40.776279 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b945e49e-c1ea-4fd7-93b5-5796a42acaa5 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:40.786846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 447/909] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Sep 5 12:06:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:40.787554 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ccdf5-b4f5-4f55-a169-582a2b64ac48 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:40.817749 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:40.896348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ccdf5-b4f5-4f55-a169-582a2b64ac48 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:40.932466 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859465fe-aeca-436f-91cc-8f4db08672d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:40.933085 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859465fe-aeca-436f-91cc-8f4db08672d7 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:40.933359 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859465fe-aeca-436f-91cc-8f4db08672d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:40.933624 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859465fe-aeca-436f-91cc-8f4db08672d7 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:40.934930 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859465fe-aeca-436f-91cc-8f4db08672d7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:40.935240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859465fe-aeca-436f-91cc-8f4db08672d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:40.936677 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859465fe-aeca-436f-91cc-8f4db08672d7 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:40.943014 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 463/910] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Sep 5 12:06:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:40.948819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b945e49e-c1ea-4fd7-93b5-5796a42acaa5 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:40.999710 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:41.073774 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b945e49e-c1ea-4fd7-93b5-5796a42acaa5 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:41.111795 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba2bc7-9893-42d9-827b-c44b3eefefe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:41.112408 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba2bc7-9893-42d9-827b-c44b3eefefe8 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:41.112685 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba2bc7-9893-42d9-827b-c44b3eefefe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:41.112956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba2bc7-9893-42d9-827b-c44b3eefefe8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:41.113660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba2bc7-9893-42d9-827b-c44b3eefefe8 None admin] RBAC: Adding query filter params (interface=public, service_id=ab0a107766774f94b93bf7ff02c6e3a9, region_id=RegionOne) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:41.113966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba2bc7-9893-42d9-827b-c44b3eefefe8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:41.115321 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba2bc7-9893-42d9-827b-c44b3eefefe8 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:41.121228 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 448/911] 10.222.0.16 () {56 vars in 1251 bytes} [Mon Sep 5 12:06:40 2022] GET /identity/v3/endpoints?service_id=ab0a107766774f94b93bf7ff02c6e3a9&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:41.126422 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859465fe-aeca-436f-91cc-8f4db08672d7 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:41.221087 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:41.232491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859465fe-aeca-436f-91cc-8f4db08672d7 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:41.269607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8efa3-612f-4852-a79e-b40eceecf046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:41.270145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8efa3-612f-4852-a79e-b40eceecf046 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:41.270422 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8efa3-612f-4852-a79e-b40eceecf046 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:41.270689 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8efa3-612f-4852-a79e-b40eceecf046 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:41.271305 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f8efa3-612f-4852-a79e-b40eceecf046 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:41.272830 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f8efa3-612f-4852-a79e-b40eceecf046 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:41.280969 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 464/912] 10.222.0.16 () {56 vars in 1087 bytes} [Mon Sep 5 12:06:41 2022] GET /identity/v3/services => generated 1497 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:06:41.881844 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50539149-c4ca-41d6-9064-226143fa2f33 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:41.882932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50539149-c4ca-41d6-9064-226143fa2f33 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:41.883454 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50539149-c4ca-41d6-9064-226143fa2f33 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:41.885505 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 449/913] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:41.891783 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de486a3c-ac8a-4f26-8897-bb88737b53e0 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:41.892382 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de486a3c-ac8a-4f26-8897-bb88737b53e0 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:41.892859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de486a3c-ac8a-4f26-8897-bb88737b53e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:41.922423 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-de486a3c-ac8a-4f26-8897-bb88737b53e0 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:41.925000 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-de486a3c-ac8a-4f26-8897-bb88737b53e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:42.009445 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 465/914] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:06:42.102404 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7eb5a9-44b1-4171-98ce-f7a879abf4ac None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:42.102793 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7eb5a9-44b1-4171-98ce-f7a879abf4ac None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:42.102978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7eb5a9-44b1-4171-98ce-f7a879abf4ac None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:42.104626 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 450/915] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:42.110632 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f3b36-7579-49e9-a2c3-2a3d3bf68055 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:42.110922 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f3b36-7579-49e9-a2c3-2a3d3bf68055 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:42.111199 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f3b36-7579-49e9-a2c3-2a3d3bf68055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:42.129986 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-d09f3b36-7579-49e9-a2c3-2a3d3bf68055 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:42.131235 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d09f3b36-7579-49e9-a2c3-2a3d3bf68055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:42.213182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 466/916] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:06:42.284837 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed86172-f4d3-41f8-9b77-4eaceb5544a1 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:42.285177 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed86172-f4d3-41f8-9b77-4eaceb5544a1 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:42.285372 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed86172-f4d3-41f8-9b77-4eaceb5544a1 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:42.286985 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 451/917] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:06:42.292379 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7383c4-7dcf-454a-bb02-94ffd7d862ed None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:42.292664 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7383c4-7dcf-454a-bb02-94ffd7d862ed None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:42.292940 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7383c4-7dcf-454a-bb02-94ffd7d862ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:42.306420 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed86172-f4d3-41f8-9b77-4eaceb5544a1 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:42.314417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-5a7383c4-7dcf-454a-bb02-94ffd7d862ed None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:42.315652 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5a7383c4-7dcf-454a-bb02-94ffd7d862ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:42.333584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:42.414515 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 467/918] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:06:42.449099 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed86172-f4d3-41f8-9b77-4eaceb5544a1 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:42.466351 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:42.499775 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101a2b7-2707-41f9-bac3-1363d0c028eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:42.500772 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101a2b7-2707-41f9-bac3-1363d0c028eb None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:42.501280 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101a2b7-2707-41f9-bac3-1363d0c028eb None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:42.501748 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101a2b7-2707-41f9-bac3-1363d0c028eb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:42.502561 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101a2b7-2707-41f9-bac3-1363d0c028eb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:42.504159 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101a2b7-2707-41f9-bac3-1363d0c028eb None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:42.513940 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-0101a2b7-2707-41f9-bac3-1363d0c028eb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 12:06:42.513940 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:42.513940 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:06:42.513940 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:06:42.513940 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:06:42.513940 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:06:42.513940 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:06:42.513940 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:42.513940 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 12:06:42.513940 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:42.513940 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:06:42.513940 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:42.513940 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:42.513940 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:42.513940 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:42.513940 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:42.513940 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:42.514972 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:42.514972 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:42.514972 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:42.514972 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:42.514972 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:42.514972 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:42.514972 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:06:42.514972 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:06:42.514972 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:06:42.514972 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:06:42.514972 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:42.514972 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:42.514972 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:42.514972 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:42.514972 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:42.514972 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:42.515846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:42.515846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:42.515846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:42.515846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:42.515846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:42.515846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:42.515846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:42.515846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:42.515846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:42.515846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:42.515846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:06:42.515846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:42.515846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 12:06:42.515846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:42.518791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 452/919] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Sep 5 12:06:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:06:42.522760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7383c4-7dcf-454a-bb02-94ffd7d862ed None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:42.525105 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101a2b7-2707-41f9-bac3-1363d0c028eb None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:42.650341 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7383c4-7dcf-454a-bb02-94ffd7d862ed None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:42.672032 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101a2b7-2707-41f9-bac3-1363d0c028eb None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:42.701411 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e59c6-c1a9-4ac8-bcaf-7cfb1773b7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:42.702080 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e59c6-c1a9-4ac8-bcaf-7cfb1773b7d2 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:42.702348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e59c6-c1a9-4ac8-bcaf-7cfb1773b7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:42.702618 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e59c6-c1a9-4ac8-bcaf-7cfb1773b7d2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:42.703318 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9e59c6-c1a9-4ac8-bcaf-7cfb1773b7d2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:42.705186 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9e59c6-c1a9-4ac8-bcaf-7cfb1773b7d2 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:42.714723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-0d9e59c6-c1a9-4ac8-bcaf-7cfb1773b7d2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 12:06:42.714723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:42.714723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:06:42.714723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:06:42.714723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:06:42.714723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:06:42.714723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:06:42.714723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:42.714723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 12:06:42.714723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:42.714723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:06:42.714723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:42.714723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:42.714723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:42.714723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:42.714723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:42.714723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:42.716406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:42.716406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:42.716406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:42.716406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:42.716406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:42.716406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:42.716406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:06:42.716406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:06:42.716406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:06:42.716406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:06:42.716406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:42.716406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:42.716406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:42.716406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:42.716406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:42.716406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:42.717682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:42.717682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:42.717682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:42.717682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:42.717682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:42.717682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:42.717682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:42.717682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:42.717682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:42.717682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:42.717682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:06:42.717682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:42.717682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 12:06:42.717682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:42.717682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 468/920] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Sep 5 12:06:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:42.718695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e59c6-c1a9-4ac8-bcaf-7cfb1773b7d2 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:42.724422 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82656a4b-72c4-4ba0-a35a-9e8bfb72463d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:42.725261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82656a4b-72c4-4ba0-a35a-9e8bfb72463d None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:42.725712 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82656a4b-72c4-4ba0-a35a-9e8bfb72463d None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:42.726128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82656a4b-72c4-4ba0-a35a-9e8bfb72463d None admin] PATH_INFO: `/v3/services` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:42.726911 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82656a4b-72c4-4ba0-a35a-9e8bfb72463d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:42.727357 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82656a4b-72c4-4ba0-a35a-9e8bfb72463d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:42.729390 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82656a4b-72c4-4ba0-a35a-9e8bfb72463d None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:42.732081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:42.736536 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 453/921] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Sep 5 12:06:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:06:42.737609 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82656a4b-72c4-4ba0-a35a-9e8bfb72463d None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:42.777299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:42.848215 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e59c6-c1a9-4ac8-bcaf-7cfb1773b7d2 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:42.858322 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82656a4b-72c4-4ba0-a35a-9e8bfb72463d None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:42.885227 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0bdb96-14ca-45b8-9011-dd651c4891aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:42.886151 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0bdb96-14ca-45b8-9011-dd651c4891aa None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:42.886588 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0bdb96-14ca-45b8-9011-dd651c4891aa None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:42.886861 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0bdb96-14ca-45b8-9011-dd651c4891aa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:42.897753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c92c9-4b13-419f-ab45-c874fe357a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:42.898306 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0bdb96-14ca-45b8-9011-dd651c4891aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:06:42.898401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c92c9-4b13-419f-ab45-c874fe357a8e None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:42.898624 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c92c9-4b13-419f-ab45-c874fe357a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:42.898895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c92c9-4b13-419f-ab45-c874fe357a8e None admin] PATH_INFO: `/v3/services` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:42.899752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c92c9-4b13-419f-ab45-c874fe357a8e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:42.900083 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c92c9-4b13-419f-ab45-c874fe357a8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:42.901428 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c92c9-4b13-419f-ab45-c874fe357a8e None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:42.903925 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-ae0bdb96-14ca-45b8-9011-dd651c4891aa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 12:06:42.903925 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:42.903925 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:42.903925 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:42.903925 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:42.903925 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:42.903925 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:42.903925 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:42.903925 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:42.903925 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:42.903925 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:42.903925 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:42.903925 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:06:42.903925 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:06:42.903925 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:06:42.903925 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:06:42.904946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:06:42.904946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:06:42.904946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:06:42.904946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:06:42.904946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:42.904946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:42.904946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:42.904946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:42.904946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:42.904946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:42.904946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:42.904946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:42.904946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:42.904946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:42.904946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:42.904946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:42.905825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:42.905825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:42.905825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:42.905825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:42.905825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:06:42.905825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:06:42.905825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:06:42.905825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:06:42.905825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:06:42.905825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:06:42.905825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 12:06:42.905825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:42.905825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 469/922] 10.222.0.16 () {56 vars in 1120 bytes} [Mon Sep 5 12:06:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:42.905825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0bdb96-14ca-45b8-9011-dd651c4891aa None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:42.910343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 454/923] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Sep 5 12:06:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:42.911056 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c92c9-4b13-419f-ab45-c874fe357a8e None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:42.986222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:43.016023 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:43.020734 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c92c9-4b13-419f-ab45-c874fe357a8e None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:43.023964 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0bdb96-14ca-45b8-9011-dd651c4891aa None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:43.058565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43bef22-f5a2-49ed-8ced-7b0db47a7c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:43.059169 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bef22-f5a2-49ed-8ced-7b0db47a7c43 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:43.059474 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bef22-f5a2-49ed-8ced-7b0db47a7c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:43.059771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bef22-f5a2-49ed-8ced-7b0db47a7c43 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:43.060432 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43bef22-f5a2-49ed-8ced-7b0db47a7c43 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:43.060729 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43bef22-f5a2-49ed-8ced-7b0db47a7c43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:43.062094 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43bef22-f5a2-49ed-8ced-7b0db47a7c43 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:43.062208 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b5a4b-6d0c-4216-ac9e-c64445881e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:43.062764 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b5a4b-6d0c-4216-ac9e-c64445881e28 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:43.063041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b5a4b-6d0c-4216-ac9e-c64445881e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:43.063482 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b5a4b-6d0c-4216-ac9e-c64445881e28 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:43.064258 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b5a4b-6d0c-4216-ac9e-c64445881e28 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:43.064566 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b5a4b-6d0c-4216-ac9e-c64445881e28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:43.065915 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b5a4b-6d0c-4216-ac9e-c64445881e28 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:43.076134 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 455/924] 10.222.0.16 () {56 vars in 1115 bytes} [Mon Sep 5 12:06:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:43.076314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 470/925] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Sep 5 12:06:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:43.076903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43bef22-f5a2-49ed-8ced-7b0db47a7c43 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:43.077111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b5a4b-6d0c-4216-ac9e-c64445881e28 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:43.183046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43bef22-f5a2-49ed-8ced-7b0db47a7c43 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:43.188453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b5a4b-6d0c-4216-ac9e-c64445881e28 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:43.219657 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:43.220219 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f315fa-2b8c-4a7c-9b33-7efb05b2e90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:43.220890 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f315fa-2b8c-4a7c-9b33-7efb05b2e90a None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:43.221179 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f315fa-2b8c-4a7c-9b33-7efb05b2e90a None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:43.221473 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f315fa-2b8c-4a7c-9b33-7efb05b2e90a None admin] PATH_INFO: `/v3/services` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:43.222345 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f315fa-2b8c-4a7c-9b33-7efb05b2e90a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:43.222685 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f315fa-2b8c-4a7c-9b33-7efb05b2e90a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:43.224155 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f315fa-2b8c-4a7c-9b33-7efb05b2e90a None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:43.226628 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e3d1c-1167-4f0e-99d5-0d387f5c2073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:43.227272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e3d1c-1167-4f0e-99d5-0d387f5c2073 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:43.227573 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e3d1c-1167-4f0e-99d5-0d387f5c2073 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:43.227850 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e3d1c-1167-4f0e-99d5-0d387f5c2073 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:43.234569 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 456/926] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Sep 5 12:06:43 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:43.234569 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f315fa-2b8c-4a7c-9b33-7efb05b2e90a None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:43.249058 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e3d1c-1167-4f0e-99d5-0d387f5c2073 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:43.250846 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e3d1c-1167-4f0e-99d5-0d387f5c2073 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:43.264662 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:43.266488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-262e3d1c-1167-4f0e-99d5-0d387f5c2073 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:43.266488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:43.266488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:43.266488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:43.266488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:43.266488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:43.266488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:43.266488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:43.266488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:43.266488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:43.266488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:43.266488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:43.266488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:43.266488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:43.266488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:43.266488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:43.271580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:43.271580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:43.271580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:43.271580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:43.271580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:43.271580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:43.271580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:43.271580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:43.271580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:43.271580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:43.271580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:43.271580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:43.271580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:43.271580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:43.271580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:43.271580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:43.271580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:43.272678 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:43.272678 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:43.272678 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:43.272678 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 471/927] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:06:43.272678 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e3d1c-1167-4f0e-99d5-0d387f5c2073 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:43.348124 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f315fa-2b8c-4a7c-9b33-7efb05b2e90a None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:43.369517 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e3d1c-1167-4f0e-99d5-0d387f5c2073 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:43.383326 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea9371-a4b7-47e5-b776-2ae5cb6851ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:43.384042 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea9371-a4b7-47e5-b776-2ae5cb6851ec None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:43.384408 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea9371-a4b7-47e5-b776-2ae5cb6851ec None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:43.384753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea9371-a4b7-47e5-b776-2ae5cb6851ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:43.385526 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea9371-a4b7-47e5-b776-2ae5cb6851ec None admin] RBAC: Adding query filter params (interface=public, service_id=2e22145824844001a2a8454745fcc0ca, region_id=RegionOne) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:43.385839 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea9371-a4b7-47e5-b776-2ae5cb6851ec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:43.387201 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea9371-a4b7-47e5-b776-2ae5cb6851ec None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:43.392933 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 457/928] 10.222.0.16 () {56 vars in 1251 bytes} [Mon Sep 5 12:06:43 2022] GET /identity/v3/endpoints?service_id=2e22145824844001a2a8454745fcc0ca&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:43.394130 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca378ac7-7a6d-49da-9702-0967dcd3cdaf None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:43.394408 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca378ac7-7a6d-49da-9702-0967dcd3cdaf None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:43.394681 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca378ac7-7a6d-49da-9702-0967dcd3cdaf None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:43.395924 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 458/929] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:43.399421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca378ac7-7a6d-49da-9702-0967dcd3cdaf None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:43.410635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbd182-4723-4834-8a56-afbb0d9811b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:43.411256 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbd182-4723-4834-8a56-afbb0d9811b6 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:43.411558 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbd182-4723-4834-8a56-afbb0d9811b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:43.411832 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbd182-4723-4834-8a56-afbb0d9811b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:43.412512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fbd182-4723-4834-8a56-afbb0d9811b6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:43.413913 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fbd182-4723-4834-8a56-afbb0d9811b6 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:43.435987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 472/930] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:06:43 2022] POST /identity/v3/endpoints => generated 346 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:06:43.436823 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbd182-4723-4834-8a56-afbb0d9811b6 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:43.484070 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:43.501253 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:43.524115 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca378ac7-7a6d-49da-9702-0967dcd3cdaf None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:43.562779 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbd182-4723-4834-8a56-afbb0d9811b6 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:43.564758 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8054d25a-6ec9-4b38-8155-1b484fbd3a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:43.565405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8054d25a-6ec9-4b38-8155-1b484fbd3a6c None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:43.565693 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8054d25a-6ec9-4b38-8155-1b484fbd3a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:43.565964 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8054d25a-6ec9-4b38-8155-1b484fbd3a6c None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:43.566636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8054d25a-6ec9-4b38-8155-1b484fbd3a6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:43.566942 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8054d25a-6ec9-4b38-8155-1b484fbd3a6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:43.568368 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8054d25a-6ec9-4b38-8155-1b484fbd3a6c None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:43.583104 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 459/931] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:43.583724 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f6f38-0f04-457c-b159-4488b3689c6d None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:43.584014 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f6f38-0f04-457c-b159-4488b3689c6d None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:43.584290 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f6f38-0f04-457c-b159-4488b3689c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:43.602642 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-ce2f6f38-0f04-457c-b159-4488b3689c6d None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:43.603880 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-ce2f6f38-0f04-457c-b159-4488b3689c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:43.607748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768c39c-f487-475f-be85-0e1b237d7d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:43.608369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768c39c-f487-475f-be85-0e1b237d7d37 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:43.608646 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768c39c-f487-475f-be85-0e1b237d7d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:43.608916 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768c39c-f487-475f-be85-0e1b237d7d37 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:43.609523 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c768c39c-f487-475f-be85-0e1b237d7d37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:43.610890 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c768c39c-f487-475f-be85-0e1b237d7d37 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:43.619574 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 473/932] 10.222.0.16 () {56 vars in 1087 bytes} [Mon Sep 5 12:06:43 2022] GET /identity/v3/services => generated 1497 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:06:43.620568 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768c39c-f487-475f-be85-0e1b237d7d37 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:43.691748 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 460/933] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:06:43.753054 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768c39c-f487-475f-be85-0e1b237d7d37 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:43.768568 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:43.792120 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3047b-4340-4100-8a2e-6ee34c7496fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:43.793122 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3047b-4340-4100-8a2e-6ee34c7496fe None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:43.794153 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3047b-4340-4100-8a2e-6ee34c7496fe None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:43.794438 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3047b-4340-4100-8a2e-6ee34c7496fe None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:43.802570 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d3047b-4340-4100-8a2e-6ee34c7496fe None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:43.804047 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d3047b-4340-4100-8a2e-6ee34c7496fe None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:43.816210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-68d3047b-4340-4100-8a2e-6ee34c7496fe None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 12:06:43.816210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:43.816210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:43.816210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:43.816210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:43.816210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:43.816210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:43.816210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:43.816210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:43.816210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:43.816210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:43.816210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:43.816210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:06:43.816210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:06:43.816210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:06:43.816210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:06:43.817265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:43.817265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:43.817265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:06:43.817265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:43.817265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:06:43.817265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:43.817265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:43.817265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:43.817265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:43.817265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:43.817265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:43.817265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:43.817265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:43.817265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:43.817265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:43.817265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:43.817265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:43.817265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:43.819304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:43.819304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:43.819304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:06:43.819304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:06:43.819304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:06:43.819304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:06:43.819304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:06:43.819304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:06:43.819304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 12:06:43.819304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:43.819304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 474/934] 10.222.0.16 () {56 vars in 1117 bytes} [Mon Sep 5 12:06:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:43.826997 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f6f38-0f04-457c-b159-4488b3689c6d None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:43.911495 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:43.939180 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f6f38-0f04-457c-b159-4488b3689c6d None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:43.974850 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6287422-d64b-4788-8b7e-b826c80e714e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:43.975466 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6287422-d64b-4788-8b7e-b826c80e714e None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:43.975783 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6287422-d64b-4788-8b7e-b826c80e714e None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:43.976055 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6287422-d64b-4788-8b7e-b826c80e714e None admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:43.976800 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6287422-d64b-4788-8b7e-b826c80e714e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:43.977123 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6287422-d64b-4788-8b7e-b826c80e714e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:43.978563 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6287422-d64b-4788-8b7e-b826c80e714e None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:43.983299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3047b-4340-4100-8a2e-6ee34c7496fe None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:44.003568 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 461/935] 10.222.0.16 () {56 vars in 1149 bytes} [Mon Sep 5 12:06:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:44.004570 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6287422-d64b-4788-8b7e-b826c80e714e None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:44.095211 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3047b-4340-4100-8a2e-6ee34c7496fe None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:44.111545 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6287422-d64b-4788-8b7e-b826c80e714e None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:44.138705 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8d056-14e4-4521-8ddd-8467f722efa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:44.139798 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8d056-14e4-4521-8ddd-8467f722efa6 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:44.140283 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8d056-14e4-4521-8ddd-8467f722efa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:44.141080 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8d056-14e4-4521-8ddd-8467f722efa6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:44.141941 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8d056-14e4-4521-8ddd-8467f722efa6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:44.143613 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8d056-14e4-4521-8ddd-8467f722efa6 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:44.153874 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed37d6-2834-4077-a66c-ebb75ae7a386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:44.154522 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed37d6-2834-4077-a66c-ebb75ae7a386 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:44.154800 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed37d6-2834-4077-a66c-ebb75ae7a386 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:44.155073 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed37d6-2834-4077-a66c-ebb75ae7a386 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:44.155369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-4aa8d056-14e4-4521-8ddd-8467f722efa6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 12:06:44.155369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:44.155369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:06:44.155369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:06:44.155369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:06:44.155369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:06:44.155369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:06:44.155369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:44.155369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 12:06:44.155369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:44.155369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:06:44.155369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:44.155369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:44.155369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:44.155369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:44.155369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:44.156601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:44.156601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:44.156601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:44.156601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:44.156601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:44.156601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:44.156601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:44.156601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:06:44.156601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:06:44.156601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:06:44.156601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:06:44.156601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:44.156601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:44.156601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:44.156601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:44.156601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:44.156601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:44.157617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:44.157617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:44.157617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:44.157617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:44.157617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:44.157617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:44.157617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:44.157617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:44.157617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:44.157617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:44.157617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:06:44.157617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:44.157617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 12:06:44.157617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:44.158843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:44.165260 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 475/936] 10.222.0.16 () {56 vars in 1132 bytes} [Mon Sep 5 12:06:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:06:44.166476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8d056-14e4-4521-8ddd-8467f722efa6 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:44.169825 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed37d6-2834-4077-a66c-ebb75ae7a386 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:44.172049 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed37d6-2834-4077-a66c-ebb75ae7a386 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:44.189906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-63ed37d6-2834-4077-a66c-ebb75ae7a386 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:44.189906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:44.189906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:44.189906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:44.189906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:44.189906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:44.189906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:44.189906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:44.189906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:44.189906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:44.189906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:44.189906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:44.189906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:44.189906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:44.189906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:44.189906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:44.190994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:44.190994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:44.190994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:44.190994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:44.190994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:44.190994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:44.190994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:44.190994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:44.190994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:44.190994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:44.190994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:44.190994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:44.190994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:44.190994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:44.190994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:44.190994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:44.190994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:44.191932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:44.191932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:44.191932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:44.195889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 462/937] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:44.198468 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed37d6-2834-4077-a66c-ebb75ae7a386 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:44.205993 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:44.307988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8d056-14e4-4521-8ddd-8467f722efa6 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:44.319011 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed37d6-2834-4077-a66c-ebb75ae7a386 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:44.354404 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1511b-b4ef-4134-8bd0-4ef94a412862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:44.354746 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688de25b-8bde-43ca-aa32-8388d76aeaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:44.355305 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688de25b-8bde-43ca-aa32-8388d76aeaf0 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:44.355609 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688de25b-8bde-43ca-aa32-8388d76aeaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:44.355928 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688de25b-8bde-43ca-aa32-8388d76aeaf0 None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:44.356890 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1511b-b4ef-4134-8bd0-4ef94a412862 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:44.357138 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688de25b-8bde-43ca-aa32-8388d76aeaf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:44.357500 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688de25b-8bde-43ca-aa32-8388d76aeaf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:44.357772 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1511b-b4ef-4134-8bd0-4ef94a412862 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:44.358237 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1511b-b4ef-4134-8bd0-4ef94a412862 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:44.358858 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688de25b-8bde-43ca-aa32-8388d76aeaf0 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:44.359279 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf1511b-b4ef-4134-8bd0-4ef94a412862 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:44.359819 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf1511b-b4ef-4134-8bd0-4ef94a412862 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:44.362076 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf1511b-b4ef-4134-8bd0-4ef94a412862 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:44.368437 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 476/938] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Sep 5 12:06:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:44.374984 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 463/939] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:06:44.379440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1511b-b4ef-4134-8bd0-4ef94a412862 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:44.380249 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688de25b-8bde-43ca-aa32-8388d76aeaf0 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:44.430632 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:44.489852 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688de25b-8bde-43ca-aa32-8388d76aeaf0 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:44.494475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:44.506018 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1511b-b4ef-4134-8bd0-4ef94a412862 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:44.527245 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969a1a5-f41f-4425-aac9-fb8f0747de9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:44.528200 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969a1a5-f41f-4425-aac9-fb8f0747de9d None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:44.528482 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969a1a5-f41f-4425-aac9-fb8f0747de9d None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:44.528744 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969a1a5-f41f-4425-aac9-fb8f0747de9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:44.536266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4969a1a5-f41f-4425-aac9-fb8f0747de9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:44.537703 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4969a1a5-f41f-4425-aac9-fb8f0747de9d None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:44.545132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df205c84-a844-4c97-9c98-f71b601870fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:44.546051 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df205c84-a844-4c97-9c98-f71b601870fb None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:44.546557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df205c84-a844-4c97-9c98-f71b601870fb None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:44.547028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df205c84-a844-4c97-9c98-f71b601870fb None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:44.547901 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df205c84-a844-4c97-9c98-f71b601870fb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:44.548413 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df205c84-a844-4c97-9c98-f71b601870fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:44.549816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-4969a1a5-f41f-4425-aac9-fb8f0747de9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:44.549816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:44.549816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:44.549816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:44.549816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:44.549816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:44.549816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:44.549816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:44.549816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:44.549816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:44.549816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:44.549816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:44.549816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:06:44.549816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:06:44.549816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:06:44.550711 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:06:44.550711 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:44.550711 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:44.550711 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:44.550711 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:44.550711 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:44.550711 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:44.550711 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:44.550711 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:44.550711 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:44.550711 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:44.550711 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:44.550711 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:44.550711 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:44.550711 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:44.550711 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:44.550711 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:44.551661 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:06:44.551661 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:06:44.551661 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:06:44.551661 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:06:44.551661 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:06:44.551661 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:06:44.551661 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:44.551661 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:44.551661 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 464/940] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:06:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:44.552156 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df205c84-a844-4c97-9c98-f71b601870fb None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:44.558470 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969a1a5-f41f-4425-aac9-fb8f0747de9d None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:44.567581 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 477/941] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Sep 5 12:06:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:44.572531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df205c84-a844-4c97-9c98-f71b601870fb None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:44.670150 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969a1a5-f41f-4425-aac9-fb8f0747de9d None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:44.672611 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:44.683233 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df205c84-a844-4c97-9c98-f71b601870fb None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:44.709555 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f08004-3a61-40d4-91d9-011dbc0995a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:44.710195 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f08004-3a61-40d4-91d9-011dbc0995a6 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:44.710415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f08004-3a61-40d4-91d9-011dbc0995a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:44.710685 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f08004-3a61-40d4-91d9-011dbc0995a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:44.711383 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f08004-3a61-40d4-91d9-011dbc0995a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:44.711786 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f08004-3a61-40d4-91d9-011dbc0995a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:44.713126 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f08004-3a61-40d4-91d9-011dbc0995a6 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:44.722660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870ae2b-63db-4d3e-8604-60eac182443a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:44.723661 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870ae2b-63db-4d3e-8604-60eac182443a None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:44.724160 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870ae2b-63db-4d3e-8604-60eac182443a None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:44.724643 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870ae2b-63db-4d3e-8604-60eac182443a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:44.730167 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6870ae2b-63db-4d3e-8604-60eac182443a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:44.730300 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 465/942] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:06:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:06:44.730384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6870ae2b-63db-4d3e-8604-60eac182443a None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:44.732166 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f08004-3a61-40d4-91d9-011dbc0995a6 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:44.763289 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 478/943] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:06:44 2022] POST /identity/v3/endpoints => generated 347 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:06:44.853596 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f08004-3a61-40d4-91d9-011dbc0995a6 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:44.897281 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a9156-c032-4526-8343-08cad3379dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:44.897948 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a9156-c032-4526-8343-08cad3379dd0 None admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:44.898231 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a9156-c032-4526-8343-08cad3379dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:44.898503 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a9156-c032-4526-8343-08cad3379dd0 None admin] PATH_INFO: `/v3/projects/fa3178684d184ec880f17640d19d55b7/users/8c5ae98336c441118d01f46a50f732d7/roles/e4660ff7081a4aaa8e0d840b2e6c6dcd` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:44.900816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a9156-c032-4526-8343-08cad3379dd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=fa3178684d184ec880f17640d19d55b7, user_id=8c5ae98336c441118d01f46a50f732d7, role_id=e4660ff7081a4aaa8e0d840b2e6c6dcd)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:44.902257 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a9156-c032-4526-8343-08cad3379dd0 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:44.920083 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 466/944] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Sep 5 12:06:44 2022] PUT /identity/v3/projects/fa3178684d184ec880f17640d19d55b7/users/8c5ae98336c441118d01f46a50f732d7/roles/e4660ff7081a4aaa8e0d840b2e6c6dcd => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:06:45.483562 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c688d3-a1e0-4661-8d1d-dc45adab908e None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:45.484077 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c688d3-a1e0-4661-8d1d-dc45adab908e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:45.484157 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c688d3-a1e0-4661-8d1d-dc45adab908e None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:45.485717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 479/945] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:45.491012 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0ccb8-026e-4d7b-bbea-1b63d4e0a4bd None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:45.491299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0ccb8-026e-4d7b-bbea-1b63d4e0a4bd None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:45.491613 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0ccb8-026e-4d7b-bbea-1b63d4e0a4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:45.509953 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-a6c0ccb8-026e-4d7b-bbea-1b63d4e0a4bd None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:45.511143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-a6c0ccb8-026e-4d7b-bbea-1b63d4e0a4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:45.526204 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:45.583116 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 467/946] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:06:45.653746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a5974-169e-4d35-b098-488b3ffd503c None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:45.653994 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a5974-169e-4d35-b098-488b3ffd503c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:45.654246 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a5974-169e-4d35-b098-488b3ffd503c None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:45.655876 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 480/947] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:45.661100 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79265642-8184-4c96-91b0-c8d1cc5e2bc4 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:45.661393 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79265642-8184-4c96-91b0-c8d1cc5e2bc4 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:45.661665 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79265642-8184-4c96-91b0-c8d1cc5e2bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:45.679955 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-79265642-8184-4c96-91b0-c8d1cc5e2bc4 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:45.681152 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-79265642-8184-4c96-91b0-c8d1cc5e2bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:45.759446 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 468/948] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:06:45.855061 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a5974-169e-4d35-b098-488b3ffd503c None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:45.872750 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:45.971739 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a5974-169e-4d35-b098-488b3ffd503c None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:46.011629 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9aef7f-dc91-4010-ba27-1292b8d22375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:46.012335 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9aef7f-dc91-4010-ba27-1292b8d22375 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:46.012627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9aef7f-dc91-4010-ba27-1292b8d22375 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:46.012897 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9aef7f-dc91-4010-ba27-1292b8d22375 None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:46.013569 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9aef7f-dc91-4010-ba27-1292b8d22375 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:46.015026 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9aef7f-dc91-4010-ba27-1292b8d22375 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:46.017880 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: WARNING keystone.server.flask.common [None req-bb9aef7f-dc91-4010-ba27-1292b8d22375 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 05 12:06:46.038028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.sql.core [None req-bb9aef7f-dc91-4010-ba27-1292b8d22375 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 05 12:06:46.038028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 05 12:06:46.038028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [parameters: {'id': '927776ae109b47488e3bd958e133f7a8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 05 12:06:46.038028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81542) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 05 12:06:46.056647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-bb9aef7f-dc91-4010-ba27-1292b8d22375 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 927776ae109b47488e3bd958e133f7a8.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 927776ae109b47488e3bd958e133f7a8. Sep 05 12:06:46.056647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:46.056647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 05 12:06:46.056647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 05 12:06:46.056647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 05 12:06:46.056647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 05 12:06:46.056647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 05 12:06:46.056647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application result = self._query(query) Sep 05 12:06:46.056647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 05 12:06:46.056647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application conn.query(q) Sep 05 12:06:46.056647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 05 12:06:46.056647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 05 12:06:46.056647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 05 12:06:46.057627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application result.read() Sep 05 12:06:46.057627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 05 12:06:46.057627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 05 12:06:46.057627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 05 12:06:46.057627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 05 12:06:46.057627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 05 12:06:46.057627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 05 12:06:46.057627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 05 12:06:46.057627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 05 12:06:46.057627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 05 12:06:46.057627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:46.057627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 05 12:06:46.057627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:46.057627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:46.057627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 05 12:06:46.057627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 05 12:06:46.057627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 05 12:06:46.057627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 05 12:06:46.058644 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 05 12:06:46.058644 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application next(self.gen) Sep 05 12:06:46.058644 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 05 12:06:46.058644 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application yield resource Sep 05 12:06:46.058644 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 05 12:06:46.058644 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application next(self.gen) Sep 05 12:06:46.058644 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 05 12:06:46.058644 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self.session.rollback() Sep 05 12:06:46.058644 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 05 12:06:46.058644 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self.force_reraise() Sep 05 12:06:46.058644 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 05 12:06:46.058644 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise self.value Sep 05 12:06:46.058644 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 05 12:06:46.058644 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 05 12:06:46.058644 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 05 12:06:46.058644 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application session.commit() Sep 05 12:06:46.058644 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 05 12:06:46.061083 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 05 12:06:46.061083 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 05 12:06:46.061083 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._prepare_impl() Sep 05 12:06:46.061083 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 05 12:06:46.061083 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self.session.flush() Sep 05 12:06:46.061083 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 05 12:06:46.061083 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._flush(objects) Sep 05 12:06:46.061083 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 05 12:06:46.061083 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 05 12:06:46.061083 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 05 12:06:46.061083 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application compat.raise_( Sep 05 12:06:46.061083 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 05 12:06:46.061083 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception Sep 05 12:06:46.061083 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 05 12:06:46.061083 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application flush_context.execute() Sep 05 12:06:46.061083 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 05 12:06:46.061083 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rec.execute(self) Sep 05 12:06:46.063168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 05 12:06:46.063168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 05 12:06:46.063168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 05 12:06:46.063168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 05 12:06:46.063168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 05 12:06:46.063168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 05 12:06:46.063168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 05 12:06:46.063168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 05 12:06:46.063168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 05 12:06:46.063168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 05 12:06:46.063168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 05 12:06:46.063168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 05 12:06:46.063168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 05 12:06:46.063168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 05 12:06:46.063168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 05 12:06:46.064856 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 05 12:06:46.064856 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 05 12:06:46.064856 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception Sep 05 12:06:46.064856 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 05 12:06:46.064856 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 05 12:06:46.064856 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 05 12:06:46.064856 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 05 12:06:46.064856 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 05 12:06:46.064856 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application result = self._query(query) Sep 05 12:06:46.064856 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 05 12:06:46.064856 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application conn.query(q) Sep 05 12:06:46.064856 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 05 12:06:46.064856 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 05 12:06:46.064856 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 05 12:06:46.064856 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application result.read() Sep 05 12:06:46.064856 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 05 12:06:46.064856 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 05 12:06:46.066871 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 05 12:06:46.066871 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 05 12:06:46.066871 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 05 12:06:46.066871 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 05 12:06:46.066871 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 05 12:06:46.066871 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 05 12:06:46.066871 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 05 12:06:46.066871 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 05 12:06:46.066871 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [parameters: {'id': '927776ae109b47488e3bd958e133f7a8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 05 12:06:46.066871 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 05 12:06:46.066871 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:46.066871 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:06:46.066871 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:46.066871 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:46.066871 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 05 12:06:46.071093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 05 12:06:46.071093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 05 12:06:46.071093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 05 12:06:46.071093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 05 12:06:46.071093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:46.071093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:06:46.071093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:46.071093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:46.071093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:46.071093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:46.071093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:46.071093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:46.071093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:46.071093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:46.071093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:46.071093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:46.071093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:46.073511 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:46.073511 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 05 12:06:46.073511 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 05 12:06:46.073511 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:46.073511 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:46.073511 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 05 12:06:46.073511 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 05 12:06:46.073511 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 927776ae109b47488e3bd958e133f7a8. Sep 05 12:06:46.073511 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:46.073511 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 481/949] 10.222.0.16 () {60 vars in 1140 bytes} [Mon Sep 5 12:06:45 2022] POST /identity/v3/projects => generated 262 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:46.074846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79265642-8184-4c96-91b0-c8d1cc5e2bc4 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:46.075350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9aef7f-dc91-4010-ba27-1292b8d22375 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:46.164388 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:46.178398 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:46.200656 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79265642-8184-4c96-91b0-c8d1cc5e2bc4 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:46.205728 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9aef7f-dc91-4010-ba27-1292b8d22375 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:46.239022 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642c69c-0786-480b-9d1e-f495133fcd4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:46.239754 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642c69c-0786-480b-9d1e-f495133fcd4f None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:46.240127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642c69c-0786-480b-9d1e-f495133fcd4f None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:46.240545 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642c69c-0786-480b-9d1e-f495133fcd4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:46.248386 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642c69c-0786-480b-9d1e-f495133fcd4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:46.248569 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4a398-cbfc-4a7e-8736-f5f8f8dcfea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:46.249225 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4a398-cbfc-4a7e-8736-f5f8f8dcfea1 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:46.249507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4a398-cbfc-4a7e-8736-f5f8f8dcfea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:46.249835 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4a398-cbfc-4a7e-8736-f5f8f8dcfea1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:46.250311 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642c69c-0786-480b-9d1e-f495133fcd4f None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:46.250424 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa4a398-cbfc-4a7e-8736-f5f8f8dcfea1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:46.252377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa4a398-cbfc-4a7e-8736-f5f8f8dcfea1 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:46.262112 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-5aa4a398-cbfc-4a7e-8736-f5f8f8dcfea1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 12:06:46.262112 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:46.262112 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:06:46.262112 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:06:46.262112 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:06:46.262112 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:06:46.262112 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:06:46.262112 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:46.262112 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 12:06:46.262112 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:46.262112 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:06:46.262112 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:46.262112 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:46.262112 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:46.262112 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:46.262112 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:46.262112 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:46.263272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:46.263272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:46.263272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:46.263272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:46.263272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:46.263272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:46.263272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:06:46.263272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:06:46.263272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:06:46.263272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:06:46.263272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:46.263272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:46.263272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:46.263272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:46.263272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:46.263272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:46.264181 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:46.264181 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:46.264181 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:46.264181 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:46.264181 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:46.264181 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:46.264181 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:46.264181 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:46.264181 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:46.264181 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:46.264181 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:06:46.264181 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:46.264181 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 12:06:46.264181 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:46.264181 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 482/950] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Sep 5 12:06:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:46.269777 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-b642c69c-0786-480b-9d1e-f495133fcd4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:46.269777 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:46.269777 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:46.269777 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:46.269777 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:46.269777 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:46.269777 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:46.269777 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:46.269777 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:46.269777 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:46.269777 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:46.269777 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:46.269777 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:06:46.269777 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:06:46.269777 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:06:46.271882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:06:46.271882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:46.271882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:46.271882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:46.271882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:46.271882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:46.271882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:46.271882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:46.271882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:46.271882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:46.271882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:46.271882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:46.271882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:46.271882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:46.271882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:46.271882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:46.271882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:46.274098 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:06:46.274098 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:06:46.274098 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:06:46.274098 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:06:46.274098 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:06:46.274098 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:06:46.274098 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:46.274098 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:46.277206 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 469/951] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:06:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:46.277402 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4a398-cbfc-4a7e-8736-f5f8f8dcfea1 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:46.277828 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642c69c-0786-480b-9d1e-f495133fcd4f None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:46.405166 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642c69c-0786-480b-9d1e-f495133fcd4f None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:46.428067 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4a398-cbfc-4a7e-8736-f5f8f8dcfea1 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:46.441607 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b24716-dba9-42a7-a011-2bad98d48229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:46.442349 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b24716-dba9-42a7-a011-2bad98d48229 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:46.442740 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b24716-dba9-42a7-a011-2bad98d48229 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:46.443321 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b24716-dba9-42a7-a011-2bad98d48229 None admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:46.444025 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b24716-dba9-42a7-a011-2bad98d48229 None admin] RBAC: Adding query filter params (name=service) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:46.444428 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b24716-dba9-42a7-a011-2bad98d48229 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:46.445954 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b24716-dba9-42a7-a011-2bad98d48229 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:46.463094 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:46.467452 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:46.469565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 470/952] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Sep 5 12:06:46 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:46.474264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5ea54-e871-4cad-8f76-61a7a255b30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:46.475120 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5ea54-e871-4cad-8f76-61a7a255b30e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:46.475608 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5ea54-e871-4cad-8f76-61a7a255b30e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:46.476043 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5ea54-e871-4cad-8f76-61a7a255b30e None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:46.476872 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5ea54-e871-4cad-8f76-61a7a255b30e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:46.477536 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5ea54-e871-4cad-8f76-61a7a255b30e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:46.479465 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5ea54-e871-4cad-8f76-61a7a255b30e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:46.489685 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 483/953] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Sep 5 12:06:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:46.496320 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b24716-dba9-42a7-a011-2bad98d48229 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:46.614104 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b24716-dba9-42a7-a011-2bad98d48229 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:46.650379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27066566-7185-4917-baab-b54e562de8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:46.651261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27066566-7185-4917-baab-b54e562de8be None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:46.651723 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27066566-7185-4917-baab-b54e562de8be None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:46.652176 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27066566-7185-4917-baab-b54e562de8be None admin] PATH_INFO: `/v3/services` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:46.653643 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27066566-7185-4917-baab-b54e562de8be None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:46.653951 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27066566-7185-4917-baab-b54e562de8be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:46.655323 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27066566-7185-4917-baab-b54e562de8be None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:46.660987 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 471/954] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Sep 5 12:06:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:46.666680 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5ea54-e871-4cad-8f76-61a7a255b30e None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:46.785998 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5ea54-e871-4cad-8f76-61a7a255b30e None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:46.810337 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7717c4-b7d0-416d-974a-2899d780ef17 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:46.810713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7717c4-b7d0-416d-974a-2899d780ef17 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:46.810915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7717c4-b7d0-416d-974a-2899d780ef17 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:46.813524 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 472/955] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:06:46.817667 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2a004-8170-4f0d-ba77-b46f4c53bf9b None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:46.817957 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2a004-8170-4f0d-ba77-b46f4c53bf9b None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:46.818228 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2a004-8170-4f0d-ba77-b46f4c53bf9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:46.823146 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed13e1eb-cc7d-4402-a3db-c1c397f76966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:46.823793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13e1eb-cc7d-4402-a3db-c1c397f76966 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:46.824075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13e1eb-cc7d-4402-a3db-c1c397f76966 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:46.824543 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13e1eb-cc7d-4402-a3db-c1c397f76966 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:46.826629 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed13e1eb-cc7d-4402-a3db-c1c397f76966 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:46.828058 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed13e1eb-cc7d-4402-a3db-c1c397f76966 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:46.838254 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-cbb2a004-8170-4f0d-ba77-b46f4c53bf9b None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:46.839481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-cbb2a004-8170-4f0d-ba77-b46f4c53bf9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:46.859617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 484/956] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:06:46 2022] POST /identity/v3/endpoints => generated 349 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:06:46.932811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:46.933177 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 473/957] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:06:47.036654 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9fb95-82c1-4461-8f2c-4b256bb70069 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:47.036951 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9fb95-82c1-4461-8f2c-4b256bb70069 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:47.037164 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9fb95-82c1-4461-8f2c-4b256bb70069 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:47.038638 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 485/958] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:47.044132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e49f3-3368-4827-bf41-51a8df88fea8 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:47.044676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e49f3-3368-4827-bf41-51a8df88fea8 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:47.045147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e49f3-3368-4827-bf41-51a8df88fea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:47.065880 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-b89e49f3-3368-4827-bf41-51a8df88fea8 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:47.067392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-b89e49f3-3368-4827-bf41-51a8df88fea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:47.138497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 474/959] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:06:47.207075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9fb95-82c1-4461-8f2c-4b256bb70069 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:47.263398 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:47.314061 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9fb95-82c1-4461-8f2c-4b256bb70069 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:47.348534 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8debc34e-0885-4e99-9e3c-5704c00b13c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:47.349159 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8debc34e-0885-4e99-9e3c-5704c00b13c2 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:47.349436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8debc34e-0885-4e99-9e3c-5704c00b13c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:47.349709 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8debc34e-0885-4e99-9e3c-5704c00b13c2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:47.350287 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8debc34e-0885-4e99-9e3c-5704c00b13c2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:47.351668 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8debc34e-0885-4e99-9e3c-5704c00b13c2 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:47.360516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-8debc34e-0885-4e99-9e3c-5704c00b13c2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 12:06:47.360516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:47.360516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:06:47.360516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:06:47.360516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:06:47.360516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:06:47.360516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:06:47.360516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:47.360516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 12:06:47.360516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:47.360516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:06:47.360516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:47.360516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:47.360516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:47.360516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:47.360516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:47.360516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:47.361691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:47.361691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:47.361691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:47.361691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:47.361691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:47.361691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:47.361691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:06:47.361691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:06:47.361691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:06:47.361691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:06:47.361691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:47.361691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:47.361691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:47.361691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:47.361691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:47.361691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:47.362630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:47.362630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:47.362630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:47.362630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:47.362630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:47.362630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:47.362630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:47.362630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:47.362630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:47.362630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:47.362630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:06:47.362630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:47.362630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 12:06:47.362630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:47.366415 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 486/960] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:06:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:47.373445 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e49f3-3368-4827-bf41-51a8df88fea8 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:47.426860 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8debc34e-0885-4e99-9e3c-5704c00b13c2 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:47.503334 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e49f3-3368-4827-bf41-51a8df88fea8 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:47.531296 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:47.536720 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8debc34e-0885-4e99-9e3c-5704c00b13c2 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:47.552599 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a38f64a-5de5-4460-a833-49455c32ad73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:47.553495 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38f64a-5de5-4460-a833-49455c32ad73 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:47.553777 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38f64a-5de5-4460-a833-49455c32ad73 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:47.554058 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38f64a-5de5-4460-a833-49455c32ad73 None admin] PATH_INFO: `/v3/services` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:47.554948 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a38f64a-5de5-4460-a833-49455c32ad73 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:47.555738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a38f64a-5de5-4460-a833-49455c32ad73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:47.557404 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a38f64a-5de5-4460-a833-49455c32ad73 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:47.558156 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:47.565549 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 475/961] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Sep 5 12:06:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:47.572280 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a38f64a-5de5-4460-a833-49455c32ad73 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:47.583394 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d54ba5-0d43-4225-8bba-45dc16f30c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:47.584075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d54ba5-0d43-4225-8bba-45dc16f30c9a None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:47.584355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d54ba5-0d43-4225-8bba-45dc16f30c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:47.584631 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d54ba5-0d43-4225-8bba-45dc16f30c9a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:47.599953 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d54ba5-0d43-4225-8bba-45dc16f30c9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:06:47.608348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-46d54ba5-0d43-4225-8bba-45dc16f30c9a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 12:06:47.608348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:47.608348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:47.608348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:47.608348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:47.608348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:47.608348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:47.608348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:47.608348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:47.608348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:47.608348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:47.608348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:47.608348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:06:47.608348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:06:47.608348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:06:47.608348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:06:47.609342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:06:47.609342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:06:47.609342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:06:47.609342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:06:47.609342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:47.609342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:47.609342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:47.609342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:47.609342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:47.609342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:47.609342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:47.609342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:47.609342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:47.609342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:47.609342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:47.609342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:47.610794 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:47.610794 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:47.610794 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:47.610794 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:47.610794 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:06:47.610794 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:06:47.610794 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:06:47.610794 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:06:47.610794 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:06:47.610794 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:06:47.610794 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 12:06:47.610794 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:47.610794 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 487/962] 10.222.0.16 () {56 vars in 1120 bytes} [Mon Sep 5 12:06:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:06:47.615240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d54ba5-0d43-4225-8bba-45dc16f30c9a None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:47.712172 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a38f64a-5de5-4460-a833-49455c32ad73 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:47.738766 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d54ba5-0d43-4225-8bba-45dc16f30c9a None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:47.765248 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d90d79-2e75-4c86-805a-f4133eb49118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:47.766551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d90d79-2e75-4c86-805a-f4133eb49118 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:47.767207 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d90d79-2e75-4c86-805a-f4133eb49118 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:47.767653 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d90d79-2e75-4c86-805a-f4133eb49118 None admin] PATH_INFO: `/v3/services` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:47.768696 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d90d79-2e75-4c86-805a-f4133eb49118 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:47.769570 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d90d79-2e75-4c86-805a-f4133eb49118 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:47.771308 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d90d79-2e75-4c86-805a-f4133eb49118 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:47.783544 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 476/963] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Sep 5 12:06:47 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:47.785621 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fc673-ba01-4404-8594-d2e825782b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:47.786320 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fc673-ba01-4404-8594-d2e825782b5e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:47.786596 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fc673-ba01-4404-8594-d2e825782b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:47.786865 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fc673-ba01-4404-8594-d2e825782b5e None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:47.787674 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454fc673-ba01-4404-8594-d2e825782b5e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:47.787980 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454fc673-ba01-4404-8594-d2e825782b5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:47.789492 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454fc673-ba01-4404-8594-d2e825782b5e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:47.800828 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 488/964] 10.222.0.16 () {56 vars in 1115 bytes} [Mon Sep 5 12:06:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:06:47.808739 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d90d79-2e75-4c86-805a-f4133eb49118 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:47.866454 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:47.923622 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d90d79-2e75-4c86-805a-f4133eb49118 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:47.960044 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d0c89-a35a-4ed3-9bb9-b0acafa744e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:47.960919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d0c89-a35a-4ed3-9bb9-b0acafa744e6 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:47.961364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d0c89-a35a-4ed3-9bb9-b0acafa744e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:47.961768 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d0c89-a35a-4ed3-9bb9-b0acafa744e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:47.973213 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d0c89-a35a-4ed3-9bb9-b0acafa744e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:47.974636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d0c89-a35a-4ed3-9bb9-b0acafa744e6 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:47.990314 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-f21d0c89-a35a-4ed3-9bb9-b0acafa744e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:47.990314 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:47.990314 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:47.990314 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:47.990314 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:47.990314 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:47.990314 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:47.990314 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:47.990314 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:47.990314 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:47.990314 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:47.990314 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:47.990314 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:47.990314 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:47.990314 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:47.990314 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:47.991317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:47.991317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:47.991317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:47.991317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:47.991317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:47.991317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:47.991317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:47.991317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:47.991317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:47.991317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:47.991317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:47.991317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:47.991317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:47.991317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:47.991317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:47.991317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:47.991317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:47.992880 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:47.992880 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:47.992880 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:47.992880 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 477/965] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:47.997554 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fc673-ba01-4404-8594-d2e825782b5e None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:48.040115 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:48.105125 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fc673-ba01-4404-8594-d2e825782b5e None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:48.140644 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f0c0e-ead5-41b7-a329-60d40d326532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:48.141159 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f0c0e-ead5-41b7-a329-60d40d326532 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:48.141435 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f0c0e-ead5-41b7-a329-60d40d326532 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:48.141704 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f0c0e-ead5-41b7-a329-60d40d326532 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:48.142333 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f0c0e-ead5-41b7-a329-60d40d326532 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:48.142630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f0c0e-ead5-41b7-a329-60d40d326532 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:48.144070 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f0c0e-ead5-41b7-a329-60d40d326532 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:48.156680 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 489/966] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:48.162191 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d0c89-a35a-4ed3-9bb9-b0acafa744e6 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:48.268683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:48.272989 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d0c89-a35a-4ed3-9bb9-b0acafa744e6 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:48.317015 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f7458-9b79-4e9f-93fb-d4b251c9dc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:48.317638 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f7458-9b79-4e9f-93fb-d4b251c9dc3e None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:48.317912 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f7458-9b79-4e9f-93fb-d4b251c9dc3e None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:48.318339 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f7458-9b79-4e9f-93fb-d4b251c9dc3e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:48.326753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f7458-9b79-4e9f-93fb-d4b251c9dc3e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:48.328213 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f7458-9b79-4e9f-93fb-d4b251c9dc3e None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:48.339698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-2d4f7458-9b79-4e9f-93fb-d4b251c9dc3e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 12:06:48.339698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:48.339698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:48.339698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:48.339698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:48.339698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:48.339698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:48.339698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:48.339698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:48.339698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:48.339698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:48.339698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:48.339698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:06:48.339698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:06:48.339698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:06:48.339698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:06:48.341261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:48.341261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:48.341261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:06:48.341261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:48.341261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:06:48.341261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:48.341261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:48.341261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:48.341261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:48.341261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:48.341261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:48.341261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:48.341261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:48.341261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:48.341261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:48.341261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:48.341261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:48.341261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:48.342886 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:48.342886 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:48.342886 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:06:48.342886 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:06:48.342886 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:06:48.342886 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:06:48.342886 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:06:48.342886 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:06:48.342886 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 12:06:48.342886 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:48.342886 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 478/967] 10.222.0.16 () {56 vars in 1117 bytes} [Mon Sep 5 12:06:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:48.347421 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f0c0e-ead5-41b7-a329-60d40d326532 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:48.459404 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f0c0e-ead5-41b7-a329-60d40d326532 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:48.463877 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:48.499998 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981dfd5-ae20-44ba-98d9-004c33538012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:48.500695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981dfd5-ae20-44ba-98d9-004c33538012 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:48.501112 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981dfd5-ae20-44ba-98d9-004c33538012 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:48.501390 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981dfd5-ae20-44ba-98d9-004c33538012 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:48.502127 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7981dfd5-ae20-44ba-98d9-004c33538012 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:48.502428 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7981dfd5-ae20-44ba-98d9-004c33538012 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:48.503891 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7981dfd5-ae20-44ba-98d9-004c33538012 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:48.526081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 490/968] 10.222.0.16 () {56 vars in 1149 bytes} [Mon Sep 5 12:06:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:48.532290 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8dd11-91fc-47fb-b0fb-62b8db4be480 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:48.532580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981dfd5-ae20-44ba-98d9-004c33538012 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:48.533059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8dd11-91fc-47fb-b0fb-62b8db4be480 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:48.533561 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8dd11-91fc-47fb-b0fb-62b8db4be480 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:48.535378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 479/969] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:48.540861 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e011add-f052-4477-813e-b952fca2efc9 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:48.541457 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e011add-f052-4477-813e-b952fca2efc9 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:48.541931 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e011add-f052-4477-813e-b952fca2efc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:48.565896 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-6e011add-f052-4477-813e-b952fca2efc9 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:48.568292 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-6e011add-f052-4477-813e-b952fca2efc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:48.664101 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 480/970] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:06:48.674079 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981dfd5-ae20-44ba-98d9-004c33538012 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:48.745416 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558120cc-4609-4f6a-aef4-bb720208b581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:48.747153 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558120cc-4609-4f6a-aef4-bb720208b581 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:48.749041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558120cc-4609-4f6a-aef4-bb720208b581 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:48.749625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558120cc-4609-4f6a-aef4-bb720208b581 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:48.779804 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:48.842374 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558120cc-4609-4f6a-aef4-bb720208b581 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:48.844164 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558120cc-4609-4f6a-aef4-bb720208b581 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:48.946953 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9eb209-6d09-4e62-a26f-ff3964ed4b4c None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:48.947479 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9eb209-6d09-4e62-a26f-ff3964ed4b4c None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:48.947929 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9eb209-6d09-4e62-a26f-ff3964ed4b4c None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:48.949689 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 481/971] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:48.955805 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0082719-2e8c-4cce-8fa9-686125a91529 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:48.956116 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0082719-2e8c-4cce-8fa9-686125a91529 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:48.956435 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0082719-2e8c-4cce-8fa9-686125a91529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:48.965526 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 491/972] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:06:48 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 05 12:06:48.966388 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558120cc-4609-4f6a-aef4-bb720208b581 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:48.983448 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-d0082719-2e8c-4cce-8fa9-686125a91529 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:48.985981 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-d0082719-2e8c-4cce-8fa9-686125a91529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:49.063242 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:49.091102 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 482/973] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:06:49.091102 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0082719-2e8c-4cce-8fa9-686125a91529 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:49.116610 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:49.128754 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558120cc-4609-4f6a-aef4-bb720208b581 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:49.188345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d7a5dd-3680-43d9-abad-83ce845ea6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:49.189014 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7a5dd-3680-43d9-abad-83ce845ea6f9 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:49.189297 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7a5dd-3680-43d9-abad-83ce845ea6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:49.189679 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7a5dd-3680-43d9-abad-83ce845ea6f9 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:49.190371 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d7a5dd-3680-43d9-abad-83ce845ea6f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:49.192061 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d7a5dd-3680-43d9-abad-83ce845ea6f9 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:49.212934 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.sql.core [None req-84d7a5dd-3680-43d9-abad-83ce845ea6f9 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 05 12:06:49.212934 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 05 12:06:49.212934 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [parameters: {'user_id': '480f9458650e4e24a765a3e1cbe5b1bf', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 05 12:06:49.212934 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81542) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 05 12:06:49.219993 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0082719-2e8c-4cce-8fa9-686125a91529 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:49.227446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-84d7a5dd-3680-43d9-abad-83ce845ea6f9 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 05 12:06:49.227446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:49.227446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 05 12:06:49.227446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 05 12:06:49.227446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 05 12:06:49.227446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 05 12:06:49.227446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 05 12:06:49.227446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application result = self._query(query) Sep 05 12:06:49.227446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 05 12:06:49.227446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application conn.query(q) Sep 05 12:06:49.227446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 05 12:06:49.227446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 05 12:06:49.227446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 05 12:06:49.227446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application result.read() Sep 05 12:06:49.227446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 05 12:06:49.228410 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 05 12:06:49.228410 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 05 12:06:49.228410 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 05 12:06:49.228410 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 05 12:06:49.228410 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 05 12:06:49.228410 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 05 12:06:49.228410 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 05 12:06:49.228410 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 05 12:06:49.228410 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:49.228410 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 05 12:06:49.228410 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:49.228410 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:49.228410 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 05 12:06:49.228410 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 05 12:06:49.228410 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 05 12:06:49.228410 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 05 12:06:49.228410 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 05 12:06:49.228410 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application next(self.gen) Sep 05 12:06:49.229503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 05 12:06:49.229503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application yield resource Sep 05 12:06:49.229503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 05 12:06:49.229503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application next(self.gen) Sep 05 12:06:49.229503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 05 12:06:49.229503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self.session.rollback() Sep 05 12:06:49.229503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 05 12:06:49.229503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self.force_reraise() Sep 05 12:06:49.229503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 05 12:06:49.229503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise self.value Sep 05 12:06:49.229503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 05 12:06:49.229503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 05 12:06:49.229503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 05 12:06:49.229503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application session.commit() Sep 05 12:06:49.229503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 05 12:06:49.229503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 05 12:06:49.229503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 05 12:06:49.230418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._prepare_impl() Sep 05 12:06:49.230418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 05 12:06:49.230418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self.session.flush() Sep 05 12:06:49.230418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 05 12:06:49.230418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._flush(objects) Sep 05 12:06:49.230418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 05 12:06:49.230418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 05 12:06:49.230418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 05 12:06:49.230418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application compat.raise_( Sep 05 12:06:49.230418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 05 12:06:49.230418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception Sep 05 12:06:49.230418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 05 12:06:49.230418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application flush_context.execute() Sep 05 12:06:49.230418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 05 12:06:49.230418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rec.execute(self) Sep 05 12:06:49.230418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 05 12:06:49.230418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 05 12:06:49.232943 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 05 12:06:49.232943 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 05 12:06:49.232943 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 05 12:06:49.232943 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 05 12:06:49.232943 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 05 12:06:49.232943 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 05 12:06:49.232943 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 05 12:06:49.232943 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 05 12:06:49.232943 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 05 12:06:49.232943 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 05 12:06:49.232943 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 05 12:06:49.232943 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 05 12:06:49.232943 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 05 12:06:49.232943 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 05 12:06:49.232943 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 05 12:06:49.232943 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception Sep 05 12:06:49.234949 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 05 12:06:49.234949 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 05 12:06:49.234949 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 05 12:06:49.234949 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 05 12:06:49.234949 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 05 12:06:49.234949 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application result = self._query(query) Sep 05 12:06:49.234949 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 05 12:06:49.234949 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application conn.query(q) Sep 05 12:06:49.234949 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 05 12:06:49.234949 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 05 12:06:49.234949 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 05 12:06:49.234949 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application result.read() Sep 05 12:06:49.234949 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 05 12:06:49.234949 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 05 12:06:49.234949 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 05 12:06:49.234949 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 05 12:06:49.248031 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 05 12:06:49.248031 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 05 12:06:49.248031 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 05 12:06:49.248031 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 05 12:06:49.248031 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 05 12:06:49.248031 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 05 12:06:49.248031 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [parameters: {'user_id': '480f9458650e4e24a765a3e1cbe5b1bf', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 05 12:06:49.248031 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 05 12:06:49.248031 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:49.248031 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:06:49.248031 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:49.248031 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:49.248031 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:49.248031 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:49.248031 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:49.250089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:49.250089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:49.250089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:49.250089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:49.250089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:49.250089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:49.250089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:49.250089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 05 12:06:49.250089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 05 12:06:49.250089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:49.250089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:49.250089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:06:49.250089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:49.250089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:06:49.250089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:49.250089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 05 12:06:49.250089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 05 12:06:49.252287 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 05 12:06:49.252287 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 05 12:06:49.252287 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 05 12:06:49.252287 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 05 12:06:49.252287 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 05 12:06:49.252287 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:49.252287 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 492/974] 10.222.0.16 () {60 vars in 1131 bytes} [Mon Sep 5 12:06:48 2022] POST /identity/v3/users => generated 160 bytes in 264 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:49.252287 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d7a5dd-3680-43d9-abad-83ce845ea6f9 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:49.264353 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492a45d-aa33-48ba-81a3-164a627ca189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:49.265112 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492a45d-aa33-48ba-81a3-164a627ca189 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:49.265396 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492a45d-aa33-48ba-81a3-164a627ca189 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:49.265670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492a45d-aa33-48ba-81a3-164a627ca189 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:49.279134 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e492a45d-aa33-48ba-81a3-164a627ca189 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:49.280688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e492a45d-aa33-48ba-81a3-164a627ca189 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:49.299107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-e492a45d-aa33-48ba-81a3-164a627ca189 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:49.299107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:49.299107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:49.299107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:49.299107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:49.299107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:49.299107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:49.299107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:49.299107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:49.299107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:49.299107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:49.299107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:49.299107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:06:49.299107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:06:49.299107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:06:49.299107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:06:49.300222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:49.300222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:49.300222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:49.300222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:49.300222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:49.300222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:49.300222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:49.300222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:49.300222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:49.300222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:49.300222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:49.300222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:49.300222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:49.300222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:49.300222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:49.300222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:49.300222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:06:49.301258 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:06:49.301258 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:06:49.301258 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:06:49.301258 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 483/975] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:49.309565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492a45d-aa33-48ba-81a3-164a627ca189 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:49.401724 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d7a5dd-3680-43d9-abad-83ce845ea6f9 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:49.428035 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:49.432198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:49.456494 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9877ecb-9147-47a5-adbd-bfeb36aff4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:49.457491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9877ecb-9147-47a5-adbd-bfeb36aff4da None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:49.457943 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9877ecb-9147-47a5-adbd-bfeb36aff4da None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:49.458413 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9877ecb-9147-47a5-adbd-bfeb36aff4da None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:49.464029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492a45d-aa33-48ba-81a3-164a627ca189 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:49.468847 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9877ecb-9147-47a5-adbd-bfeb36aff4da None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:49.470791 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9877ecb-9147-47a5-adbd-bfeb36aff4da None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:49.486004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-f9877ecb-9147-47a5-adbd-bfeb36aff4da None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 12:06:49.486004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:49.486004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:49.486004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:49.486004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:49.486004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:49.486004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:49.486004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:49.486004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:49.486004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:49.486004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:49.486004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:49.486004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:06:49.486004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:06:49.486004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:06:49.486004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:06:49.487074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:49.487074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:49.487074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:06:49.487074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:49.487074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:06:49.487074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:06:49.487074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:49.487074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:49.487074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:49.487074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:49.487074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:49.487074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:49.487074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:49.487074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:49.487074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:49.487074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:49.487074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:49.487074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:49.488079 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:49.488079 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:49.488079 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:06:49.488079 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:06:49.488079 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:06:49.488079 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:06:49.488079 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:06:49.488079 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:06:49.488079 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 12:06:49.488079 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:49.494062 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 493/976] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:06:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:49.494062 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9877ecb-9147-47a5-adbd-bfeb36aff4da None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:49.513824 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f571f1-e64f-4f21-a69f-6e1a5cb80d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:49.514537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f571f1-e64f-4f21-a69f-6e1a5cb80d63 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:49.514830 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f571f1-e64f-4f21-a69f-6e1a5cb80d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:49.515104 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f571f1-e64f-4f21-a69f-6e1a5cb80d63 None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:49.515816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f571f1-e64f-4f21-a69f-6e1a5cb80d63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:49.516123 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f571f1-e64f-4f21-a69f-6e1a5cb80d63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:49.518055 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f571f1-e64f-4f21-a69f-6e1a5cb80d63 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:49.536178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 484/977] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:49.536178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f571f1-e64f-4f21-a69f-6e1a5cb80d63 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:49.633620 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9877ecb-9147-47a5-adbd-bfeb36aff4da None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:49.675723 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f571f1-e64f-4f21-a69f-6e1a5cb80d63 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:49.678624 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82868abd-5cb8-4029-846b-7824661d49f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:49.679570 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82868abd-5cb8-4029-846b-7824661d49f6 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:49.680351 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82868abd-5cb8-4029-846b-7824661d49f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:49.680351 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82868abd-5cb8-4029-846b-7824661d49f6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:49.681339 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82868abd-5cb8-4029-846b-7824661d49f6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:49.683688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82868abd-5cb8-4029-846b-7824661d49f6 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:49.697612 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-82868abd-5cb8-4029-846b-7824661d49f6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 12:06:49.697612 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:49.697612 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:06:49.697612 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:06:49.697612 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:06:49.697612 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:06:49.697612 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:06:49.697612 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:49.697612 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 12:06:49.697612 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:49.697612 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:06:49.697612 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:49.697612 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:49.697612 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:49.697612 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:49.697612 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:49.697612 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:49.699025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:49.699025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:49.699025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:49.699025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:49.699025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:49.699025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:49.699025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:06:49.699025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:06:49.699025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:06:49.699025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:06:49.699025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:49.699025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:49.699025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:49.699025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:49.699025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:49.699025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:49.700660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:49.700660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:49.700660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:49.700660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:49.700660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:49.700660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:49.700660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:49.700660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:49.700660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:49.700660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:49.700660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:06:49.700660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:49.700660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 12:06:49.700660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:49.700660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 494/978] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Sep 5 12:06:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:49.705638 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82868abd-5cb8-4029-846b-7824661d49f6 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:49.725057 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:49.732073 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4f764-2684-4194-87d6-ac9c0eb26df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:49.732759 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4f764-2684-4194-87d6-ac9c0eb26df2 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:49.733093 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4f764-2684-4194-87d6-ac9c0eb26df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:49.733227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4f764-2684-4194-87d6-ac9c0eb26df2 None admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:49.733929 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d4f764-2684-4194-87d6-ac9c0eb26df2 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:49.734227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d4f764-2684-4194-87d6-ac9c0eb26df2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:49.735700 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d4f764-2684-4194-87d6-ac9c0eb26df2 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:49.773458 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 485/979] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Sep 5 12:06:49 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:49.774328 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4f764-2684-4194-87d6-ac9c0eb26df2 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:49.792385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:49.861419 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82868abd-5cb8-4029-846b-7824661d49f6 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:49.906261 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd19095-efd4-4be6-8f6f-46120e828b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:49.906923 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd19095-efd4-4be6-8f6f-46120e828b6c None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:49.907196 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd19095-efd4-4be6-8f6f-46120e828b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:49.907466 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd19095-efd4-4be6-8f6f-46120e828b6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:49.915959 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd19095-efd4-4be6-8f6f-46120e828b6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:49.917590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd19095-efd4-4be6-8f6f-46120e828b6c None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:49.933385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-ccd19095-efd4-4be6-8f6f-46120e828b6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:49.933385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:49.933385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:49.933385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:49.933385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:49.933385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:49.933385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:49.933385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:49.933385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:49.933385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:49.933385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:49.933385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:49.933385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:06:49.933385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:06:49.933385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:06:49.935113 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:06:49.935113 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:49.935113 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:49.935113 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:49.935113 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:49.935113 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:49.935113 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:49.935113 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:49.935113 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:49.935113 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:49.935113 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:49.935113 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:49.935113 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:49.935113 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:49.935113 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:49.935113 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:49.935113 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:49.937036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:06:49.937036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:06:49.937036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:06:49.937036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:06:49.937036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:06:49.937036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:06:49.937036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:06:49.937036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:49.937036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 495/980] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:06:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:49.943907 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd19095-efd4-4be6-8f6f-46120e828b6c None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:49.990795 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4f764-2684-4194-87d6-ac9c0eb26df2 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:50.036462 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c2b29f-d4d8-462a-bcbb-8b294d9781ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:50.037839 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c2b29f-d4d8-462a-bcbb-8b294d9781ae None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:50.038537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c2b29f-d4d8-462a-bcbb-8b294d9781ae None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:50.039114 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c2b29f-d4d8-462a-bcbb-8b294d9781ae None admin] PATH_INFO: `/v3/services` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:50.040307 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c2b29f-d4d8-462a-bcbb-8b294d9781ae None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:50.040825 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c2b29f-d4d8-462a-bcbb-8b294d9781ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:50.043425 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c2b29f-d4d8-462a-bcbb-8b294d9781ae None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:50.055214 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 486/981] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Sep 5 12:06:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:50.057495 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdc4cc-7d7b-40cf-9280-52d72c8c42d9 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:50.057969 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdc4cc-7d7b-40cf-9280-52d72c8c42d9 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:50.058406 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdc4cc-7d7b-40cf-9280-52d72c8c42d9 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:50.061897 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 487/982] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:50.063721 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdc4cc-7d7b-40cf-9280-52d72c8c42d9 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:50.118355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd19095-efd4-4be6-8f6f-46120e828b6c None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:50.123481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:50.156128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:50.174679 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238eed21-b988-4512-b9cf-ad7fcff9cf80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:50.175302 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238eed21-b988-4512-b9cf-ad7fcff9cf80 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:50.175603 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238eed21-b988-4512-b9cf-ad7fcff9cf80 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:50.175882 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238eed21-b988-4512-b9cf-ad7fcff9cf80 None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:50.176589 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238eed21-b988-4512-b9cf-ad7fcff9cf80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:50.176898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238eed21-b988-4512-b9cf-ad7fcff9cf80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:50.182523 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238eed21-b988-4512-b9cf-ad7fcff9cf80 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:50.211438 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 496/983] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:06:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:50.213424 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdc4cc-7d7b-40cf-9280-52d72c8c42d9 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:50.214133 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ded846-2c5b-4fd8-ae0b-11c580adbec1 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:50.214752 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ded846-2c5b-4fd8-ae0b-11c580adbec1 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:50.215448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ded846-2c5b-4fd8-ae0b-11c580adbec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:50.239273 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-28ded846-2c5b-4fd8-ae0b-11c580adbec1 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:50.240853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-28ded846-2c5b-4fd8-ae0b-11c580adbec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:50.263069 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034be200-768d-43c7-a7e5-44c037c603d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:50.264079 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034be200-768d-43c7-a7e5-44c037c603d8 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:50.264490 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034be200-768d-43c7-a7e5-44c037c603d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:50.264892 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034be200-768d-43c7-a7e5-44c037c603d8 None admin] PATH_INFO: `/v3/services` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:50.265991 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034be200-768d-43c7-a7e5-44c037c603d8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:50.266379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034be200-768d-43c7-a7e5-44c037c603d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:50.268139 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034be200-768d-43c7-a7e5-44c037c603d8 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:50.274877 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 488/984] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Sep 5 12:06:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:06:50.276096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034be200-768d-43c7-a7e5-44c037c603d8 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:50.327698 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 497/985] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:06:50.328936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ded846-2c5b-4fd8-ae0b-11c580adbec1 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:50.423589 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034be200-768d-43c7-a7e5-44c037c603d8 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:50.459022 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:50.487443 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511fa6b5-b726-4f58-9b50-131b17e7f500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:50.488245 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511fa6b5-b726-4f58-9b50-131b17e7f500 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:50.489199 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511fa6b5-b726-4f58-9b50-131b17e7f500 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:50.489607 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511fa6b5-b726-4f58-9b50-131b17e7f500 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:50.490495 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511fa6b5-b726-4f58-9b50-131b17e7f500 None admin] RBAC: Adding query filter params (role.id=e4660ff7081a4aaa8e0d840b2e6c6dcd, scope.project.id=fa3178684d184ec880f17640d19d55b7, user.id=8c5ae98336c441118d01f46a50f732d7) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:50.491197 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511fa6b5-b726-4f58-9b50-131b17e7f500 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:50.492829 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511fa6b5-b726-4f58-9b50-131b17e7f500 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:50.494786 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:50.505747 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 489/986] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:06:50 2022] GET /identity/v3/role_assignments?user.id=8c5ae98336c441118d01f46a50f732d7&scope.project.id=fa3178684d184ec880f17640d19d55b7&role.id=e4660ff7081a4aaa8e0d840b2e6c6dcd => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:50.518965 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ded846-2c5b-4fd8-ae0b-11c580adbec1 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:50.586109 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4132919-ed07-45ed-97e1-d91f9cbb33a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:50.587129 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4132919-ed07-45ed-97e1-d91f9cbb33a9 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:50.587648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4132919-ed07-45ed-97e1-d91f9cbb33a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:50.588119 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4132919-ed07-45ed-97e1-d91f9cbb33a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:50.589385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4132919-ed07-45ed-97e1-d91f9cbb33a9 None admin] RBAC: Adding query filter params (interface=admin, service_id=2e22145824844001a2a8454745fcc0ca, region_id=RegionOne) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:50.589907 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4132919-ed07-45ed-97e1-d91f9cbb33a9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:50.592317 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4132919-ed07-45ed-97e1-d91f9cbb33a9 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:50.607241 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 498/987] 10.222.0.16 () {56 vars in 1249 bytes} [Mon Sep 5 12:06:50 2022] GET /identity/v3/endpoints?service_id=2e22145824844001a2a8454745fcc0ca&interface=admin®ion_id=RegionOne => generated 193 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:50.609135 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511fa6b5-b726-4f58-9b50-131b17e7f500 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:50.711811 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4132919-ed07-45ed-97e1-d91f9cbb33a9 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:50.800521 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511fa6b5-b726-4f58-9b50-131b17e7f500 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:50.864957 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc68c1-b26c-4edf-bb86-7a58f22857ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:50.865963 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc68c1-b26c-4edf-bb86-7a58f22857ab None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:50.866438 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc68c1-b26c-4edf-bb86-7a58f22857ab None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:50.866909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc68c1-b26c-4edf-bb86-7a58f22857ab None admin] PATH_INFO: `/v3/services` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:50.868195 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fc68c1-b26c-4edf-bb86-7a58f22857ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:50.870856 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fc68c1-b26c-4edf-bb86-7a58f22857ab None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:50.883378 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4132919-ed07-45ed-97e1-d91f9cbb33a9 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:50.891838 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 490/988] 10.222.0.16 () {56 vars in 1087 bytes} [Mon Sep 5 12:06:50 2022] GET /identity/v3/services => generated 1497 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:06:50.959536 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef547b0c-891a-4abf-bc9a-48ecd923ae2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:50.960604 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef547b0c-891a-4abf-bc9a-48ecd923ae2d None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:50.961094 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef547b0c-891a-4abf-bc9a-48ecd923ae2d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:50.961573 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef547b0c-891a-4abf-bc9a-48ecd923ae2d None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:50.962886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef547b0c-891a-4abf-bc9a-48ecd923ae2d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:50.965549 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef547b0c-891a-4abf-bc9a-48ecd923ae2d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:50.989255 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 499/989] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Sep 5 12:06:50 2022] POST /identity/v3/services => generated 242 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:06:53.219291 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519917e-522a-4cf5-b2c6-33308a813341 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:53.220011 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519917e-522a-4cf5-b2c6-33308a813341 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:53.220011 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519917e-522a-4cf5-b2c6-33308a813341 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:53.221551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 491/990] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:53.227652 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1bdc6-1192-485e-9abb-c51e41c32b8b None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:53.228222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1bdc6-1192-485e-9abb-c51e41c32b8b None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:53.228222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1bdc6-1192-485e-9abb-c51e41c32b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:53.255152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-44e1bdc6-1192-485e-9abb-c51e41c32b8b None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:53.257426 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-44e1bdc6-1192-485e-9abb-c51e41c32b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:53.280619 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:53.309608 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a4338-138c-424c-bb56-a56690ec183c None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:53.310072 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a4338-138c-424c-bb56-a56690ec183c None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:53.310583 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a4338-138c-424c-bb56-a56690ec183c None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:53.319124 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 492/991] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:53 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:06:53.322368 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622fc9ee-f93c-494a-a62f-f9c20a1f84c0 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:53.322912 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622fc9ee-f93c-494a-a62f-f9c20a1f84c0 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:53.323183 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622fc9ee-f93c-494a-a62f-f9c20a1f84c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:53.349796 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-622fc9ee-f93c-494a-a62f-f9c20a1f84c0 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:53.351815 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-622fc9ee-f93c-494a-a62f-f9c20a1f84c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:53.381876 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 500/992] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:53 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:06:53.421757 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:53.457408 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 493/993] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:53 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:06:53.764533 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1bdc6-1192-485e-9abb-c51e41c32b8b None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:53.901827 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1bdc6-1192-485e-9abb-c51e41c32b8b None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:53.905787 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca0823-e4ed-4d1c-9805-5d6d4d03df1b None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:53.906081 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca0823-e4ed-4d1c-9805-5d6d4d03df1b None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:53.906357 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca0823-e4ed-4d1c-9805-5d6d4d03df1b None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:53.908085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 494/994] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:53.913638 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab5557-cebe-407b-913b-f6c131f18784 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:53.914023 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab5557-cebe-407b-913b-f6c131f18784 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:53.914303 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab5557-cebe-407b-913b-f6c131f18784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:53.933102 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-96ab5557-cebe-407b-913b-f6c131f18784 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:53.934799 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-96ab5557-cebe-407b-913b-f6c131f18784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:53.957548 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c006ffd6-55ad-4218-b4fc-db945714b053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:53.958194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006ffd6-55ad-4218-b4fc-db945714b053 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:53.958479 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006ffd6-55ad-4218-b4fc-db945714b053 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:53.958751 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006ffd6-55ad-4218-b4fc-db945714b053 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:53.959334 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c006ffd6-55ad-4218-b4fc-db945714b053 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:53.961814 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c006ffd6-55ad-4218-b4fc-db945714b053 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:53.978981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-c006ffd6-55ad-4218-b4fc-db945714b053 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 12:06:53.978981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:53.978981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:06:53.978981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:06:53.978981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:06:53.978981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:06:53.978981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:06:53.978981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:53.978981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 12:06:53.978981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:53.978981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:06:53.978981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:53.978981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:53.978981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:53.978981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:53.978981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:53.978981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:53.980496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:53.980496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:53.980496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:53.980496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:53.980496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:53.980496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:53.980496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:06:53.980496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:06:53.980496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:06:53.980496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:06:53.980496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:53.980496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:53.980496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:53.980496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:53.980496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:53.980496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:53.981712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:53.981712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:53.981712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:53.981712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:53.981712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:53.981712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:53.981712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:53.981712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:53.981712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:53.981712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:53.981712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:06:53.981712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:53.981712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 12:06:53.981712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:53.981712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 501/995] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Sep 5 12:06:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:06:53.983353 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c006ffd6-55ad-4218-b4fc-db945714b053 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:54.000842 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:54.018990 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 495/996] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:53 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:06:54.020183 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab5557-cebe-407b-913b-f6c131f18784 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:54.164050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c006ffd6-55ad-4218-b4fc-db945714b053 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:54.174974 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab5557-cebe-407b-913b-f6c131f18784 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:54.190654 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:54.216097 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207562e2-6558-4d4f-95fb-3cba6cb60e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:54.217150 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207562e2-6558-4d4f-95fb-3cba6cb60e1e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:54.217589 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207562e2-6558-4d4f-95fb-3cba6cb60e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:54.218103 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207562e2-6558-4d4f-95fb-3cba6cb60e1e None admin] PATH_INFO: `/v3/services/image` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:54.218886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207562e2-6558-4d4f-95fb-3cba6cb60e1e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:54.221017 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207562e2-6558-4d4f-95fb-3cba6cb60e1e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:54.238350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-207562e2-6558-4d4f-95fb-3cba6cb60e1e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 12:06:54.238350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:54.238350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:06:54.238350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:06:54.238350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:06:54.238350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:06:54.238350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:06:54.238350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:54.238350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 12:06:54.238350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:54.238350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:06:54.238350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:54.238350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:54.238350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:54.238350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:54.238350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:54.238350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:54.241560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:54.241560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:54.241560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:54.241560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:54.241560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:54.241560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:54.241560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:06:54.241560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:06:54.241560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:06:54.241560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:06:54.241560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:54.241560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:54.241560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:54.241560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:54.241560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:54.241560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:54.243510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:54.243510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:54.243510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:54.243510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:54.243510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:54.243510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:54.243510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:54.243510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:54.243510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:54.243510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:54.243510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:06:54.243510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:54.243510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 12:06:54.243510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:54.243510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 502/997] 10.222.0.16 () {56 vars in 1105 bytes} [Mon Sep 5 12:06:53 2022] GET /identity/v3/services/image => generated 86 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:54.247403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207562e2-6558-4d4f-95fb-3cba6cb60e1e None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:54.254847 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b7084-8471-4a5d-933b-7914f4d35a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:54.256120 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b7084-8471-4a5d-933b-7914f4d35a2e None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:54.256420 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b7084-8471-4a5d-933b-7914f4d35a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:54.256761 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b7084-8471-4a5d-933b-7914f4d35a2e None admin] PATH_INFO: `/v3/services` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:54.257851 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b7084-8471-4a5d-933b-7914f4d35a2e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:54.258363 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b7084-8471-4a5d-933b-7914f4d35a2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:54.260947 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b7084-8471-4a5d-933b-7914f4d35a2e None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:54.269725 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 496/998] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Sep 5 12:06:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:54.278982 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b7084-8471-4a5d-933b-7914f4d35a2e None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:54.335021 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:54.386505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207562e2-6558-4d4f-95fb-3cba6cb60e1e None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:54.416028 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b7084-8471-4a5d-933b-7914f4d35a2e None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:54.431606 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb4cda-ffa4-4fe1-bc24-5d5212e55b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:54.432564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb4cda-ffa4-4fe1-bc24-5d5212e55b54 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:54.433022 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb4cda-ffa4-4fe1-bc24-5d5212e55b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:54.433479 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb4cda-ffa4-4fe1-bc24-5d5212e55b54 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:54.434973 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb4cda-ffa4-4fe1-bc24-5d5212e55b54 None admin] RBAC: Adding query filter params (name=image) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:54.435481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb4cda-ffa4-4fe1-bc24-5d5212e55b54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:54.437928 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb4cda-ffa4-4fe1-bc24-5d5212e55b54 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:54.451234 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 503/999] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:54.451234 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb4cda-ffa4-4fe1-bc24-5d5212e55b54 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:54.469152 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243736b2-2c47-44bf-8727-f40b49dd2754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:54.469813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243736b2-2c47-44bf-8727-f40b49dd2754 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:54.470121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243736b2-2c47-44bf-8727-f40b49dd2754 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:54.470432 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243736b2-2c47-44bf-8727-f40b49dd2754 None admin] PATH_INFO: `/v3/services` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:54.471111 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243736b2-2c47-44bf-8727-f40b49dd2754 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:54.471456 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243736b2-2c47-44bf-8727-f40b49dd2754 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:54.473189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243736b2-2c47-44bf-8727-f40b49dd2754 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:54.479337 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 497/1000] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Sep 5 12:06:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:06:54.480227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243736b2-2c47-44bf-8727-f40b49dd2754 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:54.529532 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:54.572138 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb4cda-ffa4-4fe1-bc24-5d5212e55b54 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:54.612298 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243736b2-2c47-44bf-8727-f40b49dd2754 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:54.623294 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d3875-589e-4bc6-90c4-65e0745c4d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:54.624083 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d3875-589e-4bc6-90c4-65e0745c4d6b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:54.624366 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d3875-589e-4bc6-90c4-65e0745c4d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:54.624659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d3875-589e-4bc6-90c4-65e0745c4d6b None admin] PATH_INFO: `/v3/services/compute` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:54.625347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d3875-589e-4bc6-90c4-65e0745c4d6b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:54.627018 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d3875-589e-4bc6-90c4-65e0745c4d6b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:54.630375 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:54.644800 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-d05d3875-589e-4bc6-90c4-65e0745c4d6b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 12:06:54.644800 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:54.644800 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:06:54.644800 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:06:54.644800 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:06:54.644800 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:06:54.644800 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:06:54.644800 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:54.644800 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 12:06:54.644800 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:54.644800 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:06:54.644800 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:54.644800 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:54.644800 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:54.644800 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:54.644800 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:54.644800 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:54.646853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:54.646853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:54.646853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:54.646853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:54.646853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:54.646853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:54.646853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:06:54.646853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:06:54.646853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:06:54.646853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:06:54.646853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:54.646853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:54.646853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:54.646853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:54.646853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:54.646853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:54.648317 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:54.648317 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:54.648317 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:54.648317 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:54.648317 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:54.648317 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:54.648317 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:54.648317 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:54.648317 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:54.648317 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:54.648317 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:06:54.648317 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:54.648317 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 12:06:54.648317 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:54.648317 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 504/1001] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:06:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:54.649839 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d3875-589e-4bc6-90c4-65e0745c4d6b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:54.661652 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a15d32-0407-426e-9a9d-70830aaf191b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:54.662379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a15d32-0407-426e-9a9d-70830aaf191b None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:54.662733 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a15d32-0407-426e-9a9d-70830aaf191b None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:54.663077 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a15d32-0407-426e-9a9d-70830aaf191b None admin] PATH_INFO: `/v3/services` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:54.663947 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a15d32-0407-426e-9a9d-70830aaf191b None admin] RBAC: Adding query filter params (type=image) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:54.664393 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a15d32-0407-426e-9a9d-70830aaf191b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:54.665910 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a15d32-0407-426e-9a9d-70830aaf191b None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:54.672687 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 498/1002] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:54 2022] GET /identity/v3/services?type=image => generated 122 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:54.673670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a15d32-0407-426e-9a9d-70830aaf191b None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:54.786086 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d3875-589e-4bc6-90c4-65e0745c4d6b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:54.812485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:54.818402 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a15d32-0407-426e-9a9d-70830aaf191b None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:54.838799 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3cf7f-1814-47bf-9f31-7d78b845dc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:54.839495 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3cf7f-1814-47bf-9f31-7d78b845dc58 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:54.839804 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3cf7f-1814-47bf-9f31-7d78b845dc58 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:54.840089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3cf7f-1814-47bf-9f31-7d78b845dc58 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:54.841429 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3cf7f-1814-47bf-9f31-7d78b845dc58 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:54.843035 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3cf7f-1814-47bf-9f31-7d78b845dc58 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:54.863104 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 505/1003] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:06:54 2022] POST /identity/v3/endpoints => generated 325 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:06:54.882097 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d84014-02ce-4f59-929f-75abfc61977c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:54.882722 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d84014-02ce-4f59-929f-75abfc61977c None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:54.882993 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d84014-02ce-4f59-929f-75abfc61977c None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:54.883327 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d84014-02ce-4f59-929f-75abfc61977c None admin] PATH_INFO: `/v3/services` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:54.884038 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d84014-02ce-4f59-929f-75abfc61977c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:54.884340 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d84014-02ce-4f59-929f-75abfc61977c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:54.885753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d84014-02ce-4f59-929f-75abfc61977c None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:54.892312 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 499/1004] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Sep 5 12:06:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:54.901079 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3cf7f-1814-47bf-9f31-7d78b845dc58 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:55.012896 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:55.078604 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3cf7f-1814-47bf-9f31-7d78b845dc58 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:55.122084 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa169db1-6b06-4588-a554-0ce14cd735e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:55.122736 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa169db1-6b06-4588-a554-0ce14cd735e4 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:55.123008 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa169db1-6b06-4588-a554-0ce14cd735e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:55.123289 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa169db1-6b06-4588-a554-0ce14cd735e4 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:55.124018 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa169db1-6b06-4588-a554-0ce14cd735e4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:55.124292 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa169db1-6b06-4588-a554-0ce14cd735e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:55.125674 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa169db1-6b06-4588-a554-0ce14cd735e4 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:55.131508 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 506/1005] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Sep 5 12:06:54 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:55.138044 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d84014-02ce-4f59-929f-75abfc61977c None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:55.254812 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d84014-02ce-4f59-929f-75abfc61977c None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:55.302762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f413db9-42f6-45f8-a165-e653fa3e4e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:55.303819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f413db9-42f6-45f8-a165-e653fa3e4e06 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:55.303819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f413db9-42f6-45f8-a165-e653fa3e4e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:55.304078 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f413db9-42f6-45f8-a165-e653fa3e4e06 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:55.304642 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f413db9-42f6-45f8-a165-e653fa3e4e06 None admin] RBAC: Adding query filter params (interface=public, service_id=aadb44fa81114933950a40cc27631b23, region_id=RegionOne) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:55.304966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f413db9-42f6-45f8-a165-e653fa3e4e06 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:55.306415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f413db9-42f6-45f8-a165-e653fa3e4e06 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:55.313250 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 500/1006] 10.222.0.16 () {56 vars in 1251 bytes} [Mon Sep 5 12:06:55 2022] GET /identity/v3/endpoints?service_id=aadb44fa81114933950a40cc27631b23&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:55.321545 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa169db1-6b06-4588-a554-0ce14cd735e4 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:55.456886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa169db1-6b06-4588-a554-0ce14cd735e4 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:55.484177 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:55.499050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f2f795-d8c4-40ed-911d-0ef99440b714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:55.499904 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2f795-d8c4-40ed-911d-0ef99440b714 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:55.500313 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2f795-d8c4-40ed-911d-0ef99440b714 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:55.500590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2f795-d8c4-40ed-911d-0ef99440b714 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:55.501241 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f2f795-d8c4-40ed-911d-0ef99440b714 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:55.502796 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f2f795-d8c4-40ed-911d-0ef99440b714 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:55.515604 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 507/1007] 10.222.0.16 () {56 vars in 1087 bytes} [Mon Sep 5 12:06:55 2022] GET /identity/v3/services => generated 1727 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:06:56.895879 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270cf1ee-5411-40ec-94eb-b1e826cba18c None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:56.896651 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270cf1ee-5411-40ec-94eb-b1e826cba18c None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:56.896651 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270cf1ee-5411-40ec-94eb-b1e826cba18c None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:56.897945 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 501/1008] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:06:56.903861 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b20e37-838f-4dd8-98e5-82ad73935b20 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:56.904618 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b20e37-838f-4dd8-98e5-82ad73935b20 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:56.904618 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b20e37-838f-4dd8-98e5-82ad73935b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:56.928333 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-72b20e37-838f-4dd8-98e5-82ad73935b20 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:56.930263 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-72b20e37-838f-4dd8-98e5-82ad73935b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:57.027108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 508/1009] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:56 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:06:57.049280 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b216c9e-a313-4e09-b6aa-66a7b2c509f4 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:57.049724 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b216c9e-a313-4e09-b6aa-66a7b2c509f4 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:57.050021 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b216c9e-a313-4e09-b6aa-66a7b2c509f4 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:57.051788 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 502/1010] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:57.057529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d19d13-fc73-48c9-9031-8a8915e41c52 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:57.058056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d19d13-fc73-48c9-9031-8a8915e41c52 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:57.058675 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d19d13-fc73-48c9-9031-8a8915e41c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:57.087002 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-97d19d13-fc73-48c9-9031-8a8915e41c52 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:57.088396 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-97d19d13-fc73-48c9-9031-8a8915e41c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:57.176179 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 509/1011] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:57 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:06:57.340342 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b216c9e-a313-4e09-b6aa-66a7b2c509f4 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:57.376360 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:57.485876 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d19d13-fc73-48c9-9031-8a8915e41c52 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:57.487621 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b216c9e-a313-4e09-b6aa-66a7b2c509f4 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:57.548388 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec806d5-a0a7-407f-85c5-0b82b7926f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:57.550027 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec806d5-a0a7-407f-85c5-0b82b7926f90 None admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:57.550218 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec806d5-a0a7-407f-85c5-0b82b7926f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:57.550667 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec806d5-a0a7-407f-85c5-0b82b7926f90 None admin] PATH_INFO: `/v3/services` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:57.551318 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec806d5-a0a7-407f-85c5-0b82b7926f90 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:57.554136 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec806d5-a0a7-407f-85c5-0b82b7926f90 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:57.554478 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:57.581703 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 503/1012] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Sep 5 12:06:57 2022] POST /identity/v3/services => generated 242 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:06:57.629468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d19d13-fc73-48c9-9031-8a8915e41c52 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:57.680391 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f87e372-f721-46be-aada-8afbaefd0a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:57.681110 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87e372-f721-46be-aada-8afbaefd0a62 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:57.681302 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87e372-f721-46be-aada-8afbaefd0a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:57.681572 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87e372-f721-46be-aada-8afbaefd0a62 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:57.682705 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f87e372-f721-46be-aada-8afbaefd0a62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:57.684777 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f87e372-f721-46be-aada-8afbaefd0a62 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:57.686811 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 510/1013] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:06:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:06:57.693339 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec806d5-a0a7-407f-85c5-0b82b7926f90 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:57.819484 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:57.841967 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec806d5-a0a7-407f-85c5-0b82b7926f90 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:57.880742 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228cad0e-2446-4ea2-871d-150e76cdd528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:57.881355 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228cad0e-2446-4ea2-871d-150e76cdd528 None admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:57.881648 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228cad0e-2446-4ea2-871d-150e76cdd528 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:57.881934 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228cad0e-2446-4ea2-871d-150e76cdd528 None admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:57.882591 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228cad0e-2446-4ea2-871d-150e76cdd528 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:57.884226 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228cad0e-2446-4ea2-871d-150e76cdd528 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:57.917605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 504/1014] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Sep 5 12:06:57 2022] POST /identity/v3/projects => generated 311 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:06:58.926906 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1067b3-d642-4187-985e-00cd93fd2e34 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:58.927468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1067b3-d642-4187-985e-00cd93fd2e34 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:58.927938 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1067b3-d642-4187-985e-00cd93fd2e34 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:58.930696 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 511/1015] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:06:58.939692 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be85de-e5a5-4016-b7b6-2520aa25a3b7 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:58.940488 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be85de-e5a5-4016-b7b6-2520aa25a3b7 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:58.940827 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be85de-e5a5-4016-b7b6-2520aa25a3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:58.973380 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-22be85de-e5a5-4016-b7b6-2520aa25a3b7 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:58.975579 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-22be85de-e5a5-4016-b7b6-2520aa25a3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:59.095574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 505/1016] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:58 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:06:59.469453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1067b3-d642-4187-985e-00cd93fd2e34 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:59.634956 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1067b3-d642-4187-985e-00cd93fd2e34 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:59.652374 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:59.689455 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52010414-6dec-4f8f-b808-1f7fc1a1bc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:59.690392 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52010414-6dec-4f8f-b808-1f7fc1a1bc51 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:59.690687 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52010414-6dec-4f8f-b808-1f7fc1a1bc51 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:59.690945 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52010414-6dec-4f8f-b808-1f7fc1a1bc51 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:59.691991 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52010414-6dec-4f8f-b808-1f7fc1a1bc51 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:59.694366 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52010414-6dec-4f8f-b808-1f7fc1a1bc51 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:59.707403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-52010414-6dec-4f8f-b808-1f7fc1a1bc51 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 12:06:59.707403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:59.707403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:06:59.707403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:06:59.707403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:06:59.707403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:06:59.707403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:06:59.707403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:59.707403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 12:06:59.707403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:59.707403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:06:59.707403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:59.707403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:06:59.707403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:06:59.707403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:06:59.707403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:06:59.707403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:06:59.709650 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:06:59.709650 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:06:59.709650 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:06:59.709650 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:06:59.709650 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:06:59.709650 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:06:59.709650 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:06:59.709650 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:06:59.709650 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:06:59.709650 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:06:59.709650 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:06:59.709650 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:06:59.709650 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:06:59.709650 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:06:59.709650 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:06:59.709650 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:06:59.712081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:06:59.712081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:06:59.712081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:06:59.712081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:06:59.712081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:06:59.712081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:06:59.712081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:06:59.712081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:06:59.712081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:06:59.712081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:06:59.712081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:06:59.712081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:06:59.712081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 12:06:59.712081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:06:59.719932 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 512/1017] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:06:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:06:59.720294 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be85de-e5a5-4016-b7b6-2520aa25a3b7 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:06:59.755911 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968521b1-6f50-467f-9a81-d717ccc054d7 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:59.756269 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968521b1-6f50-467f-9a81-d717ccc054d7 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:59.756629 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968521b1-6f50-467f-9a81-d717ccc054d7 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:59.758528 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 513/1018] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:06:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:06:59.766581 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37f6d1-f207-41ec-8a15-9cbca79efcb4 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:59.767038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37f6d1-f207-41ec-8a15-9cbca79efcb4 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:59.767492 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37f6d1-f207-41ec-8a15-9cbca79efcb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:59.795163 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-aa37f6d1-f207-41ec-8a15-9cbca79efcb4 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:06:59.796730 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-aa37f6d1-f207-41ec-8a15-9cbca79efcb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:06:59.806525 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:06:59.862621 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be85de-e5a5-4016-b7b6-2520aa25a3b7 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:06:59.892484 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 514/1019] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:06:59 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:06:59.910702 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96911056-e6d5-4b6b-a54e-f159c01ef48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:06:59.911320 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96911056-e6d5-4b6b-a54e-f159c01ef48d None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:06:59.911648 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96911056-e6d5-4b6b-a54e-f159c01ef48d None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:06:59.911906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96911056-e6d5-4b6b-a54e-f159c01ef48d None admin] PATH_INFO: `/v3/services` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:06:59.912542 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96911056-e6d5-4b6b-a54e-f159c01ef48d None admin] RBAC: Adding query filter params (name=compute) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:06:59.912839 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96911056-e6d5-4b6b-a54e-f159c01ef48d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:06:59.914221 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96911056-e6d5-4b6b-a54e-f159c01ef48d None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:06:59.921407 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 506/1020] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Sep 5 12:06:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:06:59.929041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37f6d1-f207-41ec-8a15-9cbca79efcb4 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:00.060444 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37f6d1-f207-41ec-8a15-9cbca79efcb4 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:00.065116 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:00.117429 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f07945-bbd3-4f50-9d47-2d03364a0d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:00.118271 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f07945-bbd3-4f50-9d47-2d03364a0d6c None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:00.118679 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f07945-bbd3-4f50-9d47-2d03364a0d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:00.119016 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f07945-bbd3-4f50-9d47-2d03364a0d6c None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:00.119734 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f07945-bbd3-4f50-9d47-2d03364a0d6c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:00.120158 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f07945-bbd3-4f50-9d47-2d03364a0d6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:00.122667 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f07945-bbd3-4f50-9d47-2d03364a0d6c None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:00.127735 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7790c0-644d-4675-b2f9-5e8711a821e9 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:00.128255 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7790c0-644d-4675-b2f9-5e8711a821e9 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:00.128359 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7790c0-644d-4675-b2f9-5e8711a821e9 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:00.130557 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 507/1021] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:07:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:07:00.131697 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 515/1022] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Sep 5 12:06:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:07:00.137239 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a41983-292e-4f27-a131-be0eb7849bb9 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:00.138374 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a41983-292e-4f27-a131-be0eb7849bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:00.138830 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a41983-292e-4f27-a131-be0eb7849bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:00.140713 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f07945-bbd3-4f50-9d47-2d03364a0d6c None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:00.164748 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-d3a41983-292e-4f27-a131-be0eb7849bb9 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:07:00.165981 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-d3a41983-292e-4f27-a131-be0eb7849bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:07:00.247696 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 508/1023] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:07:00 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:07:00.247696 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a41983-292e-4f27-a131-be0eb7849bb9 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:00.283497 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f07945-bbd3-4f50-9d47-2d03364a0d6c None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:00.322119 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:00.329223 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f5d31-7629-4839-a5dc-f088b51876d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:00.330497 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f5d31-7629-4839-a5dc-f088b51876d9 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:00.331095 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f5d31-7629-4839-a5dc-f088b51876d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:00.331633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f5d31-7629-4839-a5dc-f088b51876d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:00.332562 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f5d31-7629-4839-a5dc-f088b51876d9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:00.334155 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f5d31-7629-4839-a5dc-f088b51876d9 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:00.375729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 516/1024] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:07:00 2022] POST /identity/v3/endpoints => generated 334 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:07:00.383599 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a41983-292e-4f27-a131-be0eb7849bb9 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:00.450603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db7b46-6aec-468d-a4f0-265fdc5f0286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:00.451679 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db7b46-6aec-468d-a4f0-265fdc5f0286 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:00.452146 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db7b46-6aec-468d-a4f0-265fdc5f0286 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:00.452690 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db7b46-6aec-468d-a4f0-265fdc5f0286 None admin] PATH_INFO: `/v3/services/image` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:00.453496 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db7b46-6aec-468d-a4f0-265fdc5f0286 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:00.455323 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db7b46-6aec-468d-a4f0-265fdc5f0286 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:00.466481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-a6db7b46-6aec-468d-a4f0-265fdc5f0286 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 12:07:00.466481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:07:00.466481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:07:00.466481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:07:00.466481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:07:00.466481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:07:00.466481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:07:00.466481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:07:00.466481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 12:07:00.466481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:07:00.466481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:07:00.466481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:07:00.466481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:07:00.466481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:07:00.466481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:07:00.466481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:07:00.466481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:07:00.467646 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:07:00.467646 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:07:00.467646 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:07:00.467646 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:07:00.467646 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:07:00.467646 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:07:00.467646 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:07:00.467646 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:07:00.467646 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:07:00.467646 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:07:00.467646 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:07:00.467646 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:07:00.467646 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:07:00.467646 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:07:00.467646 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:07:00.467646 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:07:00.468537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:07:00.468537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:07:00.468537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:07:00.468537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:07:00.468537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:07:00.468537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:07:00.468537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:07:00.468537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:07:00.468537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:07:00.468537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:07:00.468537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:07:00.468537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:07:00.468537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 12:07:00.468537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:07:00.471573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 509/1025] 10.222.0.16 () {56 vars in 1105 bytes} [Mon Sep 5 12:07:00 2022] GET /identity/v3/services/image => generated 86 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:07:00.476289 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f5d31-7629-4839-a5dc-f088b51876d9 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:00.511190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:00.548569 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db7b46-6aec-468d-a4f0-265fdc5f0286 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:00.634790 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f5d31-7629-4839-a5dc-f088b51876d9 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:00.688312 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db7b46-6aec-468d-a4f0-265fdc5f0286 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:00.692339 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bfd1cb-20d6-4dce-b2d9-b6f34ce6c477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:00.692955 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfd1cb-20d6-4dce-b2d9-b6f34ce6c477 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:00.693229 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfd1cb-20d6-4dce-b2d9-b6f34ce6c477 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:00.693496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfd1cb-20d6-4dce-b2d9-b6f34ce6c477 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:00.694137 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bfd1cb-20d6-4dce-b2d9-b6f34ce6c477 None admin] RBAC: Adding query filter params (name=image) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:00.694463 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bfd1cb-20d6-4dce-b2d9-b6f34ce6c477 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:00.695847 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bfd1cb-20d6-4dce-b2d9-b6f34ce6c477 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:00.707763 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 517/1026] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:07:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:07:00.711123 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bfd1cb-20d6-4dce-b2d9-b6f34ce6c477 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:00.711986 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:00.730267 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba123a4e-1aca-4ef9-99cb-9498d3685c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:00.730950 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba123a4e-1aca-4ef9-99cb-9498d3685c62 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:00.731755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba123a4e-1aca-4ef9-99cb-9498d3685c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:00.732020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba123a4e-1aca-4ef9-99cb-9498d3685c62 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:00.733238 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba123a4e-1aca-4ef9-99cb-9498d3685c62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:00.734727 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba123a4e-1aca-4ef9-99cb-9498d3685c62 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:00.736647 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 510/1027] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:07:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:07:00.742366 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba123a4e-1aca-4ef9-99cb-9498d3685c62 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:00.823280 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:00.841199 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bfd1cb-20d6-4dce-b2d9-b6f34ce6c477 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:00.891600 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba123a4e-1aca-4ef9-99cb-9498d3685c62 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:00.900718 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef13e6c-241f-42ce-bded-d0e6bf0c2f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:00.901415 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef13e6c-241f-42ce-bded-d0e6bf0c2f1f None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:00.901745 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef13e6c-241f-42ce-bded-d0e6bf0c2f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:00.902222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef13e6c-241f-42ce-bded-d0e6bf0c2f1f None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:00.904013 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef13e6c-241f-42ce-bded-d0e6bf0c2f1f None admin] RBAC: Adding query filter params (type=image) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:00.904478 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef13e6c-241f-42ce-bded-d0e6bf0c2f1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:00.906503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef13e6c-241f-42ce-bded-d0e6bf0c2f1f None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:00.915434 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 518/1028] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:07:00 2022] GET /identity/v3/services?type=image => generated 350 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:07:00.922505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef13e6c-241f-42ce-bded-d0e6bf0c2f1f None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:00.937130 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423683ff-2371-4297-b18e-54ac81401749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:00.937817 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423683ff-2371-4297-b18e-54ac81401749 None admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:00.938128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423683ff-2371-4297-b18e-54ac81401749 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:00.938367 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423683ff-2371-4297-b18e-54ac81401749 None admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:00.939045 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423683ff-2371-4297-b18e-54ac81401749 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:00.940552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423683ff-2371-4297-b18e-54ac81401749 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:00.969329 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 511/1029] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Sep 5 12:07:00 2022] POST /identity/v3/users => generated 285 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:07:01.113038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef13e6c-241f-42ce-bded-d0e6bf0c2f1f None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:01.178822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f4d42-82a2-42d5-91ea-a9bc95665108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:01.179925 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f4d42-82a2-42d5-91ea-a9bc95665108 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:01.180436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f4d42-82a2-42d5-91ea-a9bc95665108 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:01.180921 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f4d42-82a2-42d5-91ea-a9bc95665108 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:01.182270 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f4d42-82a2-42d5-91ea-a9bc95665108 None admin] RBAC: Adding query filter params (interface=public, service_id=2f83460d572342188ef707c5d0316af7, region_id=RegionOne) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:01.182802 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f4d42-82a2-42d5-91ea-a9bc95665108 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:01.185404 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f4d42-82a2-42d5-91ea-a9bc95665108 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:01.192704 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 519/1030] 10.222.0.16 () {56 vars in 1251 bytes} [Mon Sep 5 12:07:00 2022] GET /identity/v3/endpoints?service_id=2f83460d572342188ef707c5d0316af7&interface=public®ion_id=RegionOne => generated 194 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:07:01.199432 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423683ff-2371-4297-b18e-54ac81401749 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:01.271446 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:01.364786 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423683ff-2371-4297-b18e-54ac81401749 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:01.419972 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c869f2-b3e8-4eaa-ac49-69db51f36d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:01.420635 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c869f2-b3e8-4eaa-ac49-69db51f36d7a None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:01.420932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c869f2-b3e8-4eaa-ac49-69db51f36d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:01.421178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c869f2-b3e8-4eaa-ac49-69db51f36d7a None admin] PATH_INFO: `/v3/services` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:01.421797 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c869f2-b3e8-4eaa-ac49-69db51f36d7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:01.423369 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c869f2-b3e8-4eaa-ac49-69db51f36d7a None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:01.439670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 512/1031] 10.222.0.16 () {56 vars in 1087 bytes} [Mon Sep 5 12:07:01 2022] GET /identity/v3/services => generated 1957 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:07:02.877037 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b84099-a934-4732-aa47-9217227461c4 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:02.877629 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b84099-a934-4732-aa47-9217227461c4 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:02.877629 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b84099-a934-4732-aa47-9217227461c4 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:02.879099 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 520/1032] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:07:02.884694 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb25b1-7d09-450e-949d-50803303b537 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:02.885434 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb25b1-7d09-450e-949d-50803303b537 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:02.885434 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb25b1-7d09-450e-949d-50803303b537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:02.906597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-ecfb25b1-7d09-450e-949d-50803303b537 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:07:02.908060 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-ecfb25b1-7d09-450e-949d-50803303b537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:07:03.011655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 513/1033] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:07:02 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:07:03.037163 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb28e2-ba46-4499-a321-0645fe96b5eb None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:03.037459 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb28e2-ba46-4499-a321-0645fe96b5eb None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:03.037967 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb28e2-ba46-4499-a321-0645fe96b5eb None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:03.039868 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 521/1034] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:07:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:07:03.047511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb69e5-8635-44ac-a76e-516080e00d55 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:03.047862 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb69e5-8635-44ac-a76e-516080e00d55 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:03.048135 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb69e5-8635-44ac-a76e-516080e00d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:03.070627 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-70eb69e5-8635-44ac-a76e-516080e00d55 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:07:03.071874 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-70eb69e5-8635-44ac-a76e-516080e00d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:07:03.109498 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:03.162027 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 514/1035] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:07:03 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:07:03.293160 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fb28e2-ba46-4499-a321-0645fe96b5eb None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:03.315512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:03.433336 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fb28e2-ba46-4499-a321-0645fe96b5eb None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:03.465349 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb69e5-8635-44ac-a76e-516080e00d55 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:03.480711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a39fc-2f1d-41c7-b91d-8cd149a1389e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:03.481962 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a39fc-2f1d-41c7-b91d-8cd149a1389e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:03.482355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a39fc-2f1d-41c7-b91d-8cd149a1389e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:03.482792 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a39fc-2f1d-41c7-b91d-8cd149a1389e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:03.495643 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a39fc-2f1d-41c7-b91d-8cd149a1389e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:07:03.501499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-920a39fc-2f1d-41c7-b91d-8cd149a1389e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 12:07:03.501499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:07:03.501499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:07:03.501499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:07:03.501499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:07:03.501499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:07:03.501499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:07:03.501499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:07:03.501499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:07:03.501499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:07:03.501499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:07:03.501499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:07:03.501499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:07:03.501499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:07:03.501499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:07:03.501499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:07:03.503896 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:07:03.503896 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:07:03.503896 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:07:03.503896 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:07:03.503896 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:07:03.503896 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:07:03.503896 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:07:03.503896 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:07:03.503896 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:07:03.503896 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:07:03.503896 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:07:03.503896 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:07:03.503896 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:07:03.503896 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:07:03.503896 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:07:03.503896 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:07:03.505977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:07:03.505977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:07:03.505977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:07:03.505977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:07:03.505977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:07:03.505977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:07:03.505977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:07:03.505977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:07:03.505977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:07:03.505977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:07:03.505977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 12:07:03.505977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:07:03.505977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 522/1036] 10.222.0.16 () {56 vars in 1120 bytes} [Mon Sep 5 12:07:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:07:03.505977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d8c0d-88b9-4fce-9b6e-aade98711dfa None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:03.507846 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d8c0d-88b9-4fce-9b6e-aade98711dfa None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:03.507846 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d8c0d-88b9-4fce-9b6e-aade98711dfa None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:03.507846 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 523/1037] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:07:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:07:03.508584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d8c0d-88b9-4fce-9b6e-aade98711dfa None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:03.593438 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb69e5-8635-44ac-a76e-516080e00d55 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:03.597317 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:03.625723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d8c0d-88b9-4fce-9b6e-aade98711dfa None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:03.632412 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de71517-f03d-4d32-8b2a-4d6eabb36572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:03.633076 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de71517-f03d-4d32-8b2a-4d6eabb36572 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:03.633350 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de71517-f03d-4d32-8b2a-4d6eabb36572 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:03.633624 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de71517-f03d-4d32-8b2a-4d6eabb36572 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:03.645042 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de71517-f03d-4d32-8b2a-4d6eabb36572 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:07:03.650175 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-2de71517-f03d-4d32-8b2a-4d6eabb36572 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:07:03.650175 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:07:03.650175 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:07:03.650175 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:07:03.650175 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:07:03.650175 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:07:03.650175 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:07:03.650175 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:07:03.650175 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:07:03.650175 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:07:03.650175 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:07:03.650175 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:07:03.650175 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:07:03.650175 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:07:03.650175 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:07:03.650175 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:07:03.651487 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:07:03.651487 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:07:03.651487 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:07:03.651487 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:07:03.651487 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:07:03.651487 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:07:03.651487 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:07:03.651487 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:07:03.651487 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:07:03.651487 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:07:03.651487 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:07:03.651487 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:07:03.651487 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:07:03.651487 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:07:03.651487 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:07:03.651487 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:07:03.652567 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:07:03.652567 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:07:03.652567 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:07:03.652567 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:07:03.652567 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:07:03.652567 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:07:03.652567 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:07:03.652567 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:07:03.652567 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:07:03.652567 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:07:03.652567 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:07:03.652567 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:07:03.652567 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 515/1038] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:07:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:07:03.654130 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e5886-24d2-40a0-9d07-c040661d61d3 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:03.654213 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e5886-24d2-40a0-9d07-c040661d61d3 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:03.654499 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e5886-24d2-40a0-9d07-c040661d61d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:03.669463 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978f1e0-0bd4-488f-896d-62e1c7a9eb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:03.670040 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978f1e0-0bd4-488f-896d-62e1c7a9eb62 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:03.670312 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978f1e0-0bd4-488f-896d-62e1c7a9eb62 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:03.670581 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978f1e0-0bd4-488f-896d-62e1c7a9eb62 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:03.671236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978f1e0-0bd4-488f-896d-62e1c7a9eb62 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:03.671576 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978f1e0-0bd4-488f-896d-62e1c7a9eb62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:03.672861 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-f74e5886-24d2-40a0-9d07-c040661d61d3 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:07:03.672977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978f1e0-0bd4-488f-896d-62e1c7a9eb62 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:03.674060 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-f74e5886-24d2-40a0-9d07-c040661d61d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:07:03.681781 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 524/1039] 10.222.0.16 () {56 vars in 1115 bytes} [Mon Sep 5 12:07:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:07:03.682607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978f1e0-0bd4-488f-896d-62e1c7a9eb62 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:03.697330 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:03.752692 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 516/1040] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:07:03 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:07:03.753550 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e5886-24d2-40a0-9d07-c040661d61d3 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:03.803088 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978f1e0-0bd4-488f-896d-62e1c7a9eb62 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:03.845904 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707880f5-2a36-49ae-b7a8-4f2f174e2b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:03.846531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707880f5-2a36-49ae-b7a8-4f2f174e2b90 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:03.846811 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707880f5-2a36-49ae-b7a8-4f2f174e2b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:03.847086 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707880f5-2a36-49ae-b7a8-4f2f174e2b90 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:03.847869 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707880f5-2a36-49ae-b7a8-4f2f174e2b90 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:03.848162 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707880f5-2a36-49ae-b7a8-4f2f174e2b90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:03.849543 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707880f5-2a36-49ae-b7a8-4f2f174e2b90 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:03.855841 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:03.860485 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 525/1041] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:07:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:03.868817 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707880f5-2a36-49ae-b7a8-4f2f174e2b90 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:03.918515 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e5886-24d2-40a0-9d07-c040661d61d3 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:03.961393 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a96577-1881-4036-804a-c9ce4786d3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:03.962179 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a96577-1881-4036-804a-c9ce4786d3f7 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:03.962594 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a96577-1881-4036-804a-c9ce4786d3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:03.962918 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a96577-1881-4036-804a-c9ce4786d3f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:03.978325 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a96577-1881-4036-804a-c9ce4786d3f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:03.979813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a96577-1881-4036-804a-c9ce4786d3f7 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:03.995636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-50a96577-1881-4036-804a-c9ce4786d3f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:07:03.995636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:07:03.995636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:07:03.995636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:07:03.995636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:07:03.995636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:07:03.995636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:07:03.995636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:07:03.995636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:07:03.995636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:07:03.995636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:07:03.995636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:07:03.995636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:07:03.995636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:07:03.995636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:07:03.995636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:07:03.996755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:07:03.996755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:07:03.996755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:07:03.996755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:07:03.996755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:07:03.996755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:07:03.996755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:07:03.996755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:07:03.996755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:07:03.996755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:07:03.996755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:07:03.996755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:07:03.996755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:07:03.996755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:07:03.996755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:07:03.996755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:07:03.996755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:07:03.997699 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:07:03.997699 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:07:03.997699 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:07:03.997699 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 517/1042] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:07:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:07:03.999629 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707880f5-2a36-49ae-b7a8-4f2f174e2b90 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:04.002651 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a96577-1881-4036-804a-c9ce4786d3f7 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:04.015733 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:04.042504 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca7dce-dca2-4b98-8964-4cee7c829c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:04.044744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca7dce-dca2-4b98-8964-4cee7c829c6c None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:04.045032 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca7dce-dca2-4b98-8964-4cee7c829c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:04.045347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca7dce-dca2-4b98-8964-4cee7c829c6c None admin] PATH_INFO: `/v3/users/9a8cf6fc9d9e43b8a4a63f8615d4684d` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:04.061063 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca7dce-dca2-4b98-8964-4cee7c829c6c None admin] RBAC: Authorizing `identity:get_user(user_id=9a8cf6fc9d9e43b8a4a63f8615d4684d)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:04.062626 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca7dce-dca2-4b98-8964-4cee7c829c6c None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:04.064434 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 526/1043] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:07:03 2022] GET /identity/v3/users/9a8cf6fc9d9e43b8a4a63f8615d4684d => generated 285 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:07:04.065204 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca7dce-dca2-4b98-8964-4cee7c829c6c None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:04.134611 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a96577-1881-4036-804a-c9ce4786d3f7 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:04.144750 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:04.177593 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e571b-9b96-41b9-a648-f9bdcbecafc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:04.178197 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e571b-9b96-41b9-a648-f9bdcbecafc0 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:04.178497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e571b-9b96-41b9-a648-f9bdcbecafc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:04.178791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e571b-9b96-41b9-a648-f9bdcbecafc0 None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:04.179598 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e571b-9b96-41b9-a648-f9bdcbecafc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:04.179943 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e571b-9b96-41b9-a648-f9bdcbecafc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:04.181489 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e571b-9b96-41b9-a648-f9bdcbecafc0 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:04.194479 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 518/1044] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:07:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:04.195388 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e571b-9b96-41b9-a648-f9bdcbecafc0 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:04.197482 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca7dce-dca2-4b98-8964-4cee7c829c6c None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:04.244252 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd4662-bf74-45bb-9b97-dc3e46e1c175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:04.245632 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd4662-bf74-45bb-9b97-dc3e46e1c175 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:04.246181 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd4662-bf74-45bb-9b97-dc3e46e1c175 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:04.246838 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd4662-bf74-45bb-9b97-dc3e46e1c175 None admin] PATH_INFO: `/v3/services/image` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:04.248031 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd4662-bf74-45bb-9b97-dc3e46e1c175 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:04.250819 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd4662-bf74-45bb-9b97-dc3e46e1c175 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:04.264377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-d2cd4662-bf74-45bb-9b97-dc3e46e1c175 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 12:07:04.264377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:07:04.264377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:07:04.264377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:07:04.264377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:07:04.264377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:07:04.264377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:07:04.264377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:07:04.264377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 12:07:04.264377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:07:04.264377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:07:04.264377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:07:04.264377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:07:04.264377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:07:04.264377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:07:04.264377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:07:04.264377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:07:04.266011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:07:04.266011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:07:04.266011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:07:04.266011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:07:04.266011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:07:04.266011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:07:04.266011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:07:04.266011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:07:04.266011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:07:04.266011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:07:04.266011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:07:04.266011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:07:04.266011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:07:04.266011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:07:04.266011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:07:04.266011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:07:04.267190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:07:04.267190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:07:04.267190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:07:04.267190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:07:04.267190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:07:04.267190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:07:04.267190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:07:04.267190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:07:04.267190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:07:04.267190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:07:04.267190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:07:04.267190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:07:04.267190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 12:07:04.267190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:07:04.271640 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:04.272218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 527/1045] 10.222.0.16 () {56 vars in 1105 bytes} [Mon Sep 5 12:07:04 2022] GET /identity/v3/services/image => generated 86 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:07:04.274636 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd4662-bf74-45bb-9b97-dc3e46e1c175 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:04.312457 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e571b-9b96-41b9-a648-f9bdcbecafc0 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:04.349929 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f067d-7e75-41ca-b4bb-a561a47e21f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:04.350553 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f067d-7e75-41ca-b4bb-a561a47e21f2 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:04.350835 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f067d-7e75-41ca-b4bb-a561a47e21f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:04.351105 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f067d-7e75-41ca-b4bb-a561a47e21f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:04.399698 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd4662-bf74-45bb-9b97-dc3e46e1c175 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:04.425724 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:04.429439 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f067d-7e75-41ca-b4bb-a561a47e21f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:04.431189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f067d-7e75-41ca-b4bb-a561a47e21f2 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:04.440260 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239123f1-bb86-4b7d-b059-4ab5085a4d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:04.441003 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239123f1-bb86-4b7d-b059-4ab5085a4d43 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:04.441330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239123f1-bb86-4b7d-b059-4ab5085a4d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:04.441635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239123f1-bb86-4b7d-b059-4ab5085a4d43 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:04.451828 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239123f1-bb86-4b7d-b059-4ab5085a4d43 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:04.453684 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239123f1-bb86-4b7d-b059-4ab5085a4d43 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:04.467222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-239123f1-bb86-4b7d-b059-4ab5085a4d43 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:07:04.467222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:07:04.467222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:07:04.467222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:07:04.467222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:07:04.467222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:07:04.467222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:07:04.467222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:07:04.467222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:07:04.467222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:07:04.467222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:07:04.467222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:07:04.467222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:07:04.467222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:07:04.467222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:07:04.467222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:07:04.468529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:07:04.468529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:07:04.468529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:07:04.468529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:07:04.468529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:07:04.468529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:07:04.468529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:07:04.468529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:07:04.468529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:07:04.468529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:07:04.468529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:07:04.468529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:07:04.468529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:07:04.468529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:07:04.468529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:07:04.468529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:07:04.468529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:07:04.468529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:07:04.469531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:07:04.469531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:07:04.469531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:07:04.469531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:07:04.469531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:07:04.469531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:07:04.469531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:07:04.469531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:07:04.469531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:07:04.469531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:07:04.469531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 528/1046] 10.222.0.16 () {56 vars in 1093 bytes} [Mon Sep 5 12:07:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:07:04.469531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239123f1-bb86-4b7d-b059-4ab5085a4d43 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:04.517535 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:04.555767 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 519/1047] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:07:04 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:07:04.556641 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f067d-7e75-41ca-b4bb-a561a47e21f2 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:04.590504 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239123f1-bb86-4b7d-b059-4ab5085a4d43 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:04.629645 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2305aa6-250b-41c1-b01a-fb8d5efa2d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:04.630262 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2305aa6-250b-41c1-b01a-fb8d5efa2d23 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:04.630542 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2305aa6-250b-41c1-b01a-fb8d5efa2d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:04.630817 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2305aa6-250b-41c1-b01a-fb8d5efa2d23 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:04.631448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2305aa6-250b-41c1-b01a-fb8d5efa2d23 None admin] RBAC: Adding query filter params (name=image) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:04.631784 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2305aa6-250b-41c1-b01a-fb8d5efa2d23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:04.633156 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2305aa6-250b-41c1-b01a-fb8d5efa2d23 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:04.639235 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 529/1048] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:07:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:04.640085 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2305aa6-250b-41c1-b01a-fb8d5efa2d23 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:04.676001 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f067d-7e75-41ca-b4bb-a561a47e21f2 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:04.688915 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:04.710249 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fedb0-e969-4ef9-a80a-4b612862f1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:04.710845 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fedb0-e969-4ef9-a80a-4b612862f1fa None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:04.711117 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fedb0-e969-4ef9-a80a-4b612862f1fa None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:04.711386 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fedb0-e969-4ef9-a80a-4b612862f1fa None admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:04.712137 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386fedb0-e969-4ef9-a80a-4b612862f1fa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:04.712439 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386fedb0-e969-4ef9-a80a-4b612862f1fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:04.713840 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386fedb0-e969-4ef9-a80a-4b612862f1fa None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:04.730670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 520/1049] 10.222.0.16 () {56 vars in 1133 bytes} [Mon Sep 5 12:07:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:04.732020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fedb0-e969-4ef9-a80a-4b612862f1fa None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:04.750819 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2305aa6-250b-41c1-b01a-fb8d5efa2d23 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:04.767132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:04.797975 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d23919-afa3-41a5-9d39-5b818d477492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:04.798643 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d23919-afa3-41a5-9d39-5b818d477492 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:04.798918 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d23919-afa3-41a5-9d39-5b818d477492 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:04.799227 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d23919-afa3-41a5-9d39-5b818d477492 None admin] PATH_INFO: `/v3/projects/c6cb339307b14a88b35ef07cc196e492` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:04.800417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d23919-afa3-41a5-9d39-5b818d477492 None admin] RBAC: Authorizing `identity:get_project(project_id=c6cb339307b14a88b35ef07cc196e492)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:04.801846 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d23919-afa3-41a5-9d39-5b818d477492 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:04.803506 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 530/1050] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:07:04 2022] GET /identity/v3/projects/c6cb339307b14a88b35ef07cc196e492 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:04.804297 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d23919-afa3-41a5-9d39-5b818d477492 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:04.855158 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fedb0-e969-4ef9-a80a-4b612862f1fa None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:04.899255 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2280e-16b6-4956-8d8e-00a50c3caaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:04.899901 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2280e-16b6-4956-8d8e-00a50c3caaa9 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:04.900176 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2280e-16b6-4956-8d8e-00a50c3caaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:04.900445 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2280e-16b6-4956-8d8e-00a50c3caaa9 None admin] PATH_INFO: `/v3/services` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:04.901082 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e2280e-16b6-4956-8d8e-00a50c3caaa9 None admin] RBAC: Adding query filter params (type=image) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:04.901383 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e2280e-16b6-4956-8d8e-00a50c3caaa9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:04.903011 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e2280e-16b6-4956-8d8e-00a50c3caaa9 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:04.910461 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 521/1051] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:07:04 2022] GET /identity/v3/services?type=image => generated 350 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:04.911387 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2280e-16b6-4956-8d8e-00a50c3caaa9 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:04.928071 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d23919-afa3-41a5-9d39-5b818d477492 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:04.946385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:04.975083 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedcec0e-9a7d-41a0-94ba-2385c2495632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:04.976011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedcec0e-9a7d-41a0-94ba-2385c2495632 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:04.976468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedcec0e-9a7d-41a0-94ba-2385c2495632 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:04.976896 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedcec0e-9a7d-41a0-94ba-2385c2495632 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:05.024352 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:05.037471 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2280e-16b6-4956-8d8e-00a50c3caaa9 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:05.059678 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedcec0e-9a7d-41a0-94ba-2385c2495632 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:05.061064 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedcec0e-9a7d-41a0-94ba-2385c2495632 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:05.088594 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e2cb0-a238-4f6c-9734-38195fb3810d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:05.089251 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e2cb0-a238-4f6c-9734-38195fb3810d None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:05.089539 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e2cb0-a238-4f6c-9734-38195fb3810d None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:05.089815 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e2cb0-a238-4f6c-9734-38195fb3810d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:05.091172 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e2cb0-a238-4f6c-9734-38195fb3810d None admin] RBAC: Adding query filter params (role.id=618c48706f7f445abeddf7eb2f8e927a, scope.project.id=c6cb339307b14a88b35ef07cc196e492, user.id=9a8cf6fc9d9e43b8a4a63f8615d4684d) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:05.091484 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e2cb0-a238-4f6c-9734-38195fb3810d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:05.093032 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e2cb0-a238-4f6c-9734-38195fb3810d None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:05.099004 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 522/1052] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:07:04 2022] GET /identity/v3/role_assignments?user.id=9a8cf6fc9d9e43b8a4a63f8615d4684d&scope.project.id=c6cb339307b14a88b35ef07cc196e492&role.id=618c48706f7f445abeddf7eb2f8e927a => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:05.099913 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e2cb0-a238-4f6c-9734-38195fb3810d None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:05.219674 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 531/1053] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:07:04 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:07:05.226786 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedcec0e-9a7d-41a0-94ba-2385c2495632 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:05.248761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:05.254824 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e2cb0-a238-4f6c-9734-38195fb3810d None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:05.310555 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473754e-91a6-4226-9431-d0ecf9bb2573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:05.311315 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473754e-91a6-4226-9431-d0ecf9bb2573 None admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:05.311731 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473754e-91a6-4226-9431-d0ecf9bb2573 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:05.312191 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473754e-91a6-4226-9431-d0ecf9bb2573 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:05.312981 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473754e-91a6-4226-9431-d0ecf9bb2573 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:05.314452 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473754e-91a6-4226-9431-d0ecf9bb2573 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:05.338500 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 523/1054] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:07:05 2022] POST /identity/v3/endpoints => generated 327 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:07:05.384184 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedcec0e-9a7d-41a0-94ba-2385c2495632 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:05.435688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bb766-a250-4278-8c22-4c827398487e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:05.436344 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bb766-a250-4278-8c22-4c827398487e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:05.436627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bb766-a250-4278-8c22-4c827398487e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:05.436893 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bb766-a250-4278-8c22-4c827398487e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:05.450301 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976bb766-a250-4278-8c22-4c827398487e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:05.451908 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976bb766-a250-4278-8c22-4c827398487e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:05.470601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-976bb766-a250-4278-8c22-4c827398487e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:07:05.470601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:07:05.470601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:07:05.470601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:07:05.470601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:07:05.470601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:07:05.470601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:07:05.470601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:07:05.470601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:07:05.470601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:07:05.470601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:07:05.470601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:07:05.470601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:07:05.470601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:07:05.470601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:07:05.470601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:07:05.472140 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:07:05.472140 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:07:05.472140 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:07:05.472140 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:07:05.472140 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:07:05.472140 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:07:05.472140 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:07:05.472140 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:07:05.472140 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:07:05.472140 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:07:05.472140 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:07:05.472140 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:07:05.472140 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:07:05.472140 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:07:05.472140 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:07:05.472140 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:07:05.472140 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:07:05.473465 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:07:05.473465 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:07:05.473465 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:07:05.473465 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 532/1055] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:07:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:07:05.479670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473754e-91a6-4226-9431-d0ecf9bb2573 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:05.493919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:05.609322 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473754e-91a6-4226-9431-d0ecf9bb2573 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:05.650906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21259f-1346-4d78-a9cc-feea2f6bf119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:05.651616 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21259f-1346-4d78-a9cc-feea2f6bf119 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:05.651902 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21259f-1346-4d78-a9cc-feea2f6bf119 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:05.652176 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21259f-1346-4d78-a9cc-feea2f6bf119 None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:05.653708 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a21259f-1346-4d78-a9cc-feea2f6bf119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:05.654018 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a21259f-1346-4d78-a9cc-feea2f6bf119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:05.655582 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a21259f-1346-4d78-a9cc-feea2f6bf119 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:05.670097 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 524/1056] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:07:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:05.676057 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bb766-a250-4278-8c22-4c827398487e None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:05.747363 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:05.799388 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bb766-a250-4278-8c22-4c827398487e None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:05.844084 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b7f5a-af04-4eac-ac43-7bd4a0cc1dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:05.844875 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b7f5a-af04-4eac-ac43-7bd4a0cc1dd9 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:05.845149 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b7f5a-af04-4eac-ac43-7bd4a0cc1dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:05.845577 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b7f5a-af04-4eac-ac43-7bd4a0cc1dd9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:05.854515 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b7f5a-af04-4eac-ac43-7bd4a0cc1dd9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:05.856242 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b7f5a-af04-4eac-ac43-7bd4a0cc1dd9 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:05.872400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-7a4b7f5a-af04-4eac-ac43-7bd4a0cc1dd9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:07:05.872400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:07:05.872400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:07:05.872400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:07:05.872400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:07:05.872400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:07:05.872400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:07:05.872400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:07:05.872400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:07:05.872400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:07:05.872400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:07:05.872400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:07:05.872400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:07:05.872400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:07:05.872400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:07:05.873774 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:07:05.873774 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:07:05.873774 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:07:05.873774 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:07:05.873774 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:07:05.873774 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:07:05.873774 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:07:05.873774 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:07:05.873774 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:07:05.873774 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:07:05.873774 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:07:05.873774 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:07:05.873774 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:07:05.873774 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:07:05.873774 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:07:05.873774 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:07:05.873774 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:07:05.874739 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:07:05.874739 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:07:05.874739 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:07:05.874739 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:07:05.874739 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:07:05.874739 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:07:05.874739 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:07:05.874739 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:07:05.874739 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 533/1057] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:07:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:07:05.881697 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21259f-1346-4d78-a9cc-feea2f6bf119 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:05.983112 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:06.025382 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21259f-1346-4d78-a9cc-feea2f6bf119 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:06.080018 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9d6e7-cd20-4595-b77a-7e5cff6e21f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:06.081067 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9d6e7-cd20-4595-b77a-7e5cff6e21f3 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:06.081610 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9d6e7-cd20-4595-b77a-7e5cff6e21f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:06.082116 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9d6e7-cd20-4595-b77a-7e5cff6e21f3 None admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:06.083418 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e9d6e7-cd20-4595-b77a-7e5cff6e21f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:06.084013 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e9d6e7-cd20-4595-b77a-7e5cff6e21f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:06.086723 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e9d6e7-cd20-4595-b77a-7e5cff6e21f3 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:06.109299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 525/1058] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:07:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:07:06.115173 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b7f5a-af04-4eac-ac43-7bd4a0cc1dd9 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:06.279549 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b7f5a-af04-4eac-ac43-7bd4a0cc1dd9 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:06.305692 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:06.319812 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f394325-e5b3-463b-82fd-80f6d9ac39fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:06.320416 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f394325-e5b3-463b-82fd-80f6d9ac39fa None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:06.320693 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f394325-e5b3-463b-82fd-80f6d9ac39fa None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:06.320963 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f394325-e5b3-463b-82fd-80f6d9ac39fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:06.321687 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f394325-e5b3-463b-82fd-80f6d9ac39fa None admin] RBAC: Adding query filter params (role.id=e4660ff7081a4aaa8e0d840b2e6c6dcd, scope.project.id=fa3178684d184ec880f17640d19d55b7, user.id=7cda290418504f0db85d665479a05353) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:06.321992 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f394325-e5b3-463b-82fd-80f6d9ac39fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:06.323403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f394325-e5b3-463b-82fd-80f6d9ac39fa None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:06.335558 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 534/1059] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:07:06 2022] GET /identity/v3/role_assignments?user.id=7cda290418504f0db85d665479a05353&scope.project.id=fa3178684d184ec880f17640d19d55b7&role.id=e4660ff7081a4aaa8e0d840b2e6c6dcd => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:07.247822 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d45c8-45e3-412b-828b-2269b25c6ac7 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:07.248470 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d45c8-45e3-412b-828b-2269b25c6ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:07.248470 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d45c8-45e3-412b-828b-2269b25c6ac7 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:07.250380 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 526/1060] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:07:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:07:07.257506 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20e9f0-9438-4f18-9fd8-4f80d47bb388 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:07.258252 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20e9f0-9438-4f18-9fd8-4f80d47bb388 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:07.258252 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20e9f0-9438-4f18-9fd8-4f80d47bb388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:07.281239 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-5a20e9f0-9438-4f18-9fd8-4f80d47bb388 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:07:07.282456 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5a20e9f0-9438-4f18-9fd8-4f80d47bb388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:07:07.364977 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1942c5-1d91-4439-8f4d-1d68861a87d6 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:07.365260 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1942c5-1d91-4439-8f4d-1d68861a87d6 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:07.365560 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1942c5-1d91-4439-8f4d-1d68861a87d6 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:07.366911 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 527/1061] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:07:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:07:07.374951 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc383876-346d-4877-88d8-1fe3b34ce20a None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:07.374951 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc383876-346d-4877-88d8-1fe3b34ce20a None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:07.374951 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc383876-346d-4877-88d8-1fe3b34ce20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:07.381347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 535/1062] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:07:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:07:07.393525 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-fc383876-346d-4877-88d8-1fe3b34ce20a None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:07:07.394721 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-fc383876-346d-4877-88d8-1fe3b34ce20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:07:07.467176 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 528/1063] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:07:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:07:07.675931 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20e9f0-9438-4f18-9fd8-4f80d47bb388 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:07.734392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc383876-346d-4877-88d8-1fe3b34ce20a None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:07.805390 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20e9f0-9438-4f18-9fd8-4f80d47bb388 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:07.817635 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:07.826842 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:07.851990 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c49f3-ca14-4351-b615-2b0b084d334f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:07.852643 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c49f3-ca14-4351-b615-2b0b084d334f None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:07.857497 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c49f3-ca14-4351-b615-2b0b084d334f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:07.857497 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c49f3-ca14-4351-b615-2b0b084d334f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:07.865171 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c49f3-ca14-4351-b615-2b0b084d334f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:07:07.870326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-996c49f3-ca14-4351-b615-2b0b084d334f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:07:07.870326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:07:07.870326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:07:07.870326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:07:07.870326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:07:07.870326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:07:07.870326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:07:07.870326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:07:07.870326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:07:07.870326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:07:07.870326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:07:07.870326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:07:07.870326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:07:07.870326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:07:07.870326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:07:07.870326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:07:07.871342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:07:07.871342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:07:07.871342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:07:07.871342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:07:07.871342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:07:07.871342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:07:07.871342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:07:07.871342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:07:07.871342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:07:07.871342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:07:07.871342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:07:07.871342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:07:07.871342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:07:07.871342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:07:07.871342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:07:07.871342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:07:07.872348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:07:07.872348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:07:07.872348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:07:07.872348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:07:07.872348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:07:07.872348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:07:07.872348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:07:07.872348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:07:07.872348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:07:07.872348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:07:07.872348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:07:07.872348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:07:07.872348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 536/1064] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:07:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:07:07.879634 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c49f3-ca14-4351-b615-2b0b084d334f None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:07.891092 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc383876-346d-4877-88d8-1fe3b34ce20a None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:07.935043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2b403-7110-429e-a880-f7f8d452fbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:07.935642 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2b403-7110-429e-a880-f7f8d452fbf1 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:07.935927 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2b403-7110-429e-a880-f7f8d452fbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:07.936273 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2b403-7110-429e-a880-f7f8d452fbf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:08.017015 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2b403-7110-429e-a880-f7f8d452fbf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:08.018482 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2b403-7110-429e-a880-f7f8d452fbf1 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:08.022611 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c49f3-ca14-4351-b615-2b0b084d334f None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:08.075236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d2b84-d2b6-42d7-a3fa-f774d81c74b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:08.075889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d2b84-d2b6-42d7-a3fa-f774d81c74b4 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:08.076171 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d2b84-d2b6-42d7-a3fa-f774d81c74b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:08.076428 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d2b84-d2b6-42d7-a3fa-f774d81c74b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:08.077093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d2b84-d2b6-42d7-a3fa-f774d81c74b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:08.077407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d2b84-d2b6-42d7-a3fa-f774d81c74b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:08.078824 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d2b84-d2b6-42d7-a3fa-f774d81c74b4 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:08.088405 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 537/1065] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:07:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:07:08.095187 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d2b84-d2b6-42d7-a3fa-f774d81c74b4 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:08.096995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:08.147689 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:08.159600 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 529/1066] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:07:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:07:08.166304 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2b403-7110-429e-a880-f7f8d452fbf1 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:08.235839 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d2b84-d2b6-42d7-a3fa-f774d81c74b4 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:08.278663 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420397a-3adf-408b-a5da-1364346f10d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:08.279568 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420397a-3adf-408b-a5da-1364346f10d9 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:08.279691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420397a-3adf-408b-a5da-1364346f10d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:08.279905 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420397a-3adf-408b-a5da-1364346f10d9 None admin] PATH_INFO: `/v3/users/9a8cf6fc9d9e43b8a4a63f8615d4684d` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:08.281339 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f420397a-3adf-408b-a5da-1364346f10d9 None admin] RBAC: Authorizing `identity:get_user(user_id=9a8cf6fc9d9e43b8a4a63f8615d4684d)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:08.282693 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f420397a-3adf-408b-a5da-1364346f10d9 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:08.285273 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 538/1067] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:07:08 2022] GET /identity/v3/users/9a8cf6fc9d9e43b8a4a63f8615d4684d => generated 285 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:07:08.292715 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420397a-3adf-408b-a5da-1364346f10d9 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:08.330433 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2b403-7110-429e-a880-f7f8d452fbf1 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:08.375360 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef20a77-7b6d-4f8b-9379-c0e487d583b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:08.375914 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef20a77-7b6d-4f8b-9379-c0e487d583b8 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:08.376193 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef20a77-7b6d-4f8b-9379-c0e487d583b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:08.376462 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef20a77-7b6d-4f8b-9379-c0e487d583b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:08.388217 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef20a77-7b6d-4f8b-9379-c0e487d583b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:08.389674 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef20a77-7b6d-4f8b-9379-c0e487d583b8 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:08.407309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-5ef20a77-7b6d-4f8b-9379-c0e487d583b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:07:08.407309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:07:08.407309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:07:08.407309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:07:08.407309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:07:08.407309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:07:08.407309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:07:08.407309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:07:08.407309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:07:08.407309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:07:08.407309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:07:08.407309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:07:08.407309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:07:08.407309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:07:08.407309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:07:08.407309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:07:08.408794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:07:08.408794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:07:08.408794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:07:08.408794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:07:08.408794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:07:08.408794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:07:08.408794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:07:08.408794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:07:08.408794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:07:08.408794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:07:08.408794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:07:08.408794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:07:08.408794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:07:08.408794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:07:08.408794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:07:08.408794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:07:08.408794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:07:08.410263 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:07:08.410263 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:07:08.410263 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:07:08.410263 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 530/1068] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:07:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:07:08.410579 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd8207-aa17-437d-81ab-6412c36a2c98 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:08.410865 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd8207-aa17-437d-81ab-6412c36a2c98 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:08.411142 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd8207-aa17-437d-81ab-6412c36a2c98 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:08.412627 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 531/1069] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:07:08.416573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd8207-aa17-437d-81ab-6412c36a2c98 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:08.428183 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420397a-3adf-408b-a5da-1364346f10d9 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:08.435023 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:08.437683 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:08.470507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3faa2-c712-43e5-b828-8b54668a794f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:08.471243 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3faa2-c712-43e5-b828-8b54668a794f None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:08.471641 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3faa2-c712-43e5-b828-8b54668a794f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:08.471941 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3faa2-c712-43e5-b828-8b54668a794f None admin] PATH_INFO: `/v3/projects/c6cb339307b14a88b35ef07cc196e492` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:08.473247 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a3faa2-c712-43e5-b828-8b54668a794f None admin] RBAC: Authorizing `identity:get_project(project_id=c6cb339307b14a88b35ef07cc196e492)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:08.474947 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a3faa2-c712-43e5-b828-8b54668a794f None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:08.476836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 539/1070] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:07:08 2022] GET /identity/v3/projects/c6cb339307b14a88b35ef07cc196e492 => generated 311 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:08.483338 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70209840-abb6-444f-9bad-74cdf08a4138 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:08.483753 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70209840-abb6-444f-9bad-74cdf08a4138 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:08.484035 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70209840-abb6-444f-9bad-74cdf08a4138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:08.504436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-70209840-abb6-444f-9bad-74cdf08a4138 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:07:08.505538 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-70209840-abb6-444f-9bad-74cdf08a4138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:07:08.553202 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd8207-aa17-437d-81ab-6412c36a2c98 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:08.586635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 540/1071] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:07:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:07:08.587505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70209840-abb6-444f-9bad-74cdf08a4138 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:08.601008 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e08ff1-46f9-455e-a202-9ee58fa16d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:08.601804 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e08ff1-46f9-455e-a202-9ee58fa16d4f None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:08.602099 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e08ff1-46f9-455e-a202-9ee58fa16d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:08.602393 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e08ff1-46f9-455e-a202-9ee58fa16d4f None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:08.603039 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e08ff1-46f9-455e-a202-9ee58fa16d4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:08.603477 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e08ff1-46f9-455e-a202-9ee58fa16d4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:08.604822 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e08ff1-46f9-455e-a202-9ee58fa16d4f None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:08.627558 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 532/1072] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:08.724129 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70209840-abb6-444f-9bad-74cdf08a4138 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:08.726973 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:08.779140 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd859b09-3f67-42bd-b905-db805884d6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:08.779878 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd859b09-3f67-42bd-b905-db805884d6f9 None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:08.780165 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd859b09-3f67-42bd-b905-db805884d6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:08.780443 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd859b09-3f67-42bd-b905-db805884d6f9 None admin] PATH_INFO: `/v3/projects/c6cb339307b14a88b35ef07cc196e492/users/9a8cf6fc9d9e43b8a4a63f8615d4684d/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:08.782548 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd859b09-3f67-42bd-b905-db805884d6f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6cb339307b14a88b35ef07cc196e492, user_id=9a8cf6fc9d9e43b8a4a63f8615d4684d, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:08.784075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd859b09-3f67-42bd-b905-db805884d6f9 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:08.803222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 541/1073] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Sep 5 12:07:08 2022] PUT /identity/v3/projects/c6cb339307b14a88b35ef07cc196e492/users/9a8cf6fc9d9e43b8a4a63f8615d4684d/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:07:08.862541 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e08ff1-46f9-455e-a202-9ee58fa16d4f None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:08.972236 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:09.014111 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e08ff1-46f9-455e-a202-9ee58fa16d4f None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:09.056665 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2cc69-653b-4230-9072-2a129681b7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:09.057414 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2cc69-653b-4230-9072-2a129681b7ee None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:09.057757 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2cc69-653b-4230-9072-2a129681b7ee None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:09.058097 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2cc69-653b-4230-9072-2a129681b7ee None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:09.074110 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c2cc69-653b-4230-9072-2a129681b7ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:07:09.079537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-02c2cc69-653b-4230-9072-2a129681b7ee None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 12:07:09.079537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:07:09.079537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:07:09.079537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:07:09.079537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:07:09.079537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:07:09.079537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:07:09.079537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:07:09.079537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:07:09.079537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:07:09.079537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:07:09.079537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:07:09.079537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:07:09.079537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:07:09.079537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:07:09.079537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:07:09.081358 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:07:09.081358 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:07:09.081358 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:07:09.081358 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:07:09.081358 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:07:09.081358 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:07:09.081358 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:07:09.081358 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:07:09.081358 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:07:09.081358 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:07:09.081358 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:07:09.081358 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:07:09.081358 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:07:09.081358 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:07:09.081358 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:07:09.081358 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:07:09.083492 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:07:09.083492 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:07:09.083492 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:07:09.083492 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:07:09.083492 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:07:09.083492 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:07:09.083492 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:07:09.083492 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:07:09.083492 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:07:09.083492 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:07:09.083492 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 12:07:09.083492 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:07:09.083492 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 533/1074] 10.222.0.16 () {56 vars in 1120 bytes} [Mon Sep 5 12:07:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:07:09.086922 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd859b09-3f67-42bd-b905-db805884d6f9 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:09.209224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd859b09-3f67-42bd-b905-db805884d6f9 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:09.245622 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333aee21-13bc-4b99-a471-d5b5f9448261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:09.246392 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333aee21-13bc-4b99-a471-d5b5f9448261 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:09.246851 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333aee21-13bc-4b99-a471-d5b5f9448261 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:09.247220 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333aee21-13bc-4b99-a471-d5b5f9448261 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:09.248037 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333aee21-13bc-4b99-a471-d5b5f9448261 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:09.248425 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333aee21-13bc-4b99-a471-d5b5f9448261 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:09.249901 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333aee21-13bc-4b99-a471-d5b5f9448261 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:09.259371 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 542/1075] 10.222.0.16 () {56 vars in 1115 bytes} [Mon Sep 5 12:07:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:09.266318 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2cc69-653b-4230-9072-2a129681b7ee None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:09.379542 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2cc69-653b-4230-9072-2a129681b7ee None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:09.414164 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:09.420875 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ee0fff-438e-4877-8277-37c88fdbf6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:09.421498 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee0fff-438e-4877-8277-37c88fdbf6f5 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:09.421773 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee0fff-438e-4877-8277-37c88fdbf6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:09.422040 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee0fff-438e-4877-8277-37c88fdbf6f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:09.432943 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ee0fff-438e-4877-8277-37c88fdbf6f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:09.434396 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ee0fff-438e-4877-8277-37c88fdbf6f5 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:09.449309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-d1ee0fff-438e-4877-8277-37c88fdbf6f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:07:09.449309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:07:09.449309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:07:09.449309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:07:09.449309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:07:09.449309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:07:09.449309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:07:09.449309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:07:09.449309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:07:09.449309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:07:09.449309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:07:09.449309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:07:09.449309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:07:09.449309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:07:09.449309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:07:09.449309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:07:09.451243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:07:09.451243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:07:09.451243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:07:09.451243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:07:09.451243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:07:09.451243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:07:09.451243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:07:09.451243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:07:09.451243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:07:09.451243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:07:09.451243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:07:09.451243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:07:09.451243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:07:09.451243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:07:09.451243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:07:09.451243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:07:09.451243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:07:09.452741 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:07:09.452741 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:07:09.452741 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:07:09.452741 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 534/1076] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:07:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:07:09.457735 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333aee21-13bc-4b99-a471-d5b5f9448261 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:09.496867 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:09.587913 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333aee21-13bc-4b99-a471-d5b5f9448261 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:09.624768 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f0110-b8a6-4525-9a6f-e5deab30617e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:09.625311 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f0110-b8a6-4525-9a6f-e5deab30617e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:09.625567 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f0110-b8a6-4525-9a6f-e5deab30617e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:09.625834 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f0110-b8a6-4525-9a6f-e5deab30617e None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:09.626480 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f0110-b8a6-4525-9a6f-e5deab30617e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:09.626784 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f0110-b8a6-4525-9a6f-e5deab30617e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:09.628184 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f0110-b8a6-4525-9a6f-e5deab30617e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:09.650236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 543/1077] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:07:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:09.651553 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ee0fff-438e-4877-8277-37c88fdbf6f5 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:09.763807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ee0fff-438e-4877-8277-37c88fdbf6f5 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:09.800787 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68994244-9350-4ed9-86dc-5602a049c4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:09.801419 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68994244-9350-4ed9-86dc-5602a049c4e9 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:09.801697 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68994244-9350-4ed9-86dc-5602a049c4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:09.801974 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68994244-9350-4ed9-86dc-5602a049c4e9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:09.810972 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68994244-9350-4ed9-86dc-5602a049c4e9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:09.812468 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68994244-9350-4ed9-86dc-5602a049c4e9 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:09.825401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-68994244-9350-4ed9-86dc-5602a049c4e9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:07:09.825401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:07:09.825401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:07:09.825401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:07:09.825401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:07:09.825401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:07:09.825401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:07:09.825401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:07:09.825401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:07:09.825401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:07:09.825401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:07:09.825401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:07:09.825401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:07:09.825401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:07:09.825401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:07:09.825401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:07:09.826526 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:07:09.826526 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:07:09.826526 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:07:09.826526 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:07:09.826526 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:07:09.826526 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:07:09.826526 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:07:09.826526 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:07:09.826526 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:07:09.826526 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:07:09.826526 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:07:09.826526 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:07:09.826526 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:07:09.826526 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:07:09.826526 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:07:09.826526 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:07:09.826526 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:07:09.826526 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:07:09.827573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:07:09.827573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:07:09.827573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:07:09.827573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:07:09.827573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:07:09.827573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:07:09.827573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:07:09.827573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:07:09.827573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:07:09.827573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:07:09.827573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 535/1078] 10.222.0.16 () {56 vars in 1093 bytes} [Mon Sep 5 12:07:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:07:09.832366 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f0110-b8a6-4525-9a6f-e5deab30617e None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:09.924501 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:09.945353 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f0110-b8a6-4525-9a6f-e5deab30617e None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:09.980355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eccb0f-d22a-43f4-b546-ac0cf342afac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:09.981457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eccb0f-d22a-43f4-b546-ac0cf342afac None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:09.981728 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eccb0f-d22a-43f4-b546-ac0cf342afac None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:09.982002 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eccb0f-d22a-43f4-b546-ac0cf342afac None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:09.982724 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eccb0f-d22a-43f4-b546-ac0cf342afac None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:09.983025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eccb0f-d22a-43f4-b546-ac0cf342afac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:09.984484 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eccb0f-d22a-43f4-b546-ac0cf342afac None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:10.013570 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 544/1079] 10.222.0.16 () {56 vars in 1133 bytes} [Mon Sep 5 12:07:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:10.014961 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68994244-9350-4ed9-86dc-5602a049c4e9 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:10.070185 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:10.140010 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68994244-9350-4ed9-86dc-5602a049c4e9 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:10.182384 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d4a89-ab78-4972-837c-6884a277d3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:10.183288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d4a89-ab78-4972-837c-6884a277d3da None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:10.183606 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d4a89-ab78-4972-837c-6884a277d3da None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:10.183883 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d4a89-ab78-4972-837c-6884a277d3da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:10.196439 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39d4a89-ab78-4972-837c-6884a277d3da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:10.197899 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39d4a89-ab78-4972-837c-6884a277d3da None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:10.212771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-a39d4a89-ab78-4972-837c-6884a277d3da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:07:10.212771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:07:10.212771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:07:10.212771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:07:10.212771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:07:10.212771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:07:10.212771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:07:10.212771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:07:10.212771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:07:10.212771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:07:10.212771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:07:10.212771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:07:10.212771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:07:10.212771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:07:10.212771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:07:10.212771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:07:10.221302 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:07:10.221302 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:07:10.221302 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:07:10.221302 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:07:10.221302 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:07:10.221302 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:07:10.221302 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:07:10.221302 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:07:10.221302 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:07:10.221302 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:07:10.221302 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:07:10.221302 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:07:10.221302 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:07:10.221302 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:07:10.221302 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:07:10.221302 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:07:10.221302 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:07:10.223026 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:07:10.223026 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:07:10.223026 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:07:10.223026 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 536/1080] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:07:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:07:10.223575 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eccb0f-d22a-43f4-b546-ac0cf342afac None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:10.332059 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eccb0f-d22a-43f4-b546-ac0cf342afac None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:10.370062 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3196187e-ddd2-487e-9be6-79fd44b03def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:10.370677 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196187e-ddd2-487e-9be6-79fd44b03def None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:10.370966 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196187e-ddd2-487e-9be6-79fd44b03def None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:10.371239 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196187e-ddd2-487e-9be6-79fd44b03def None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:10.371968 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3196187e-ddd2-487e-9be6-79fd44b03def None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:10.372269 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3196187e-ddd2-487e-9be6-79fd44b03def None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:10.374310 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3196187e-ddd2-487e-9be6-79fd44b03def None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:10.385652 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 545/1081] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:07:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:10.392004 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d4a89-ab78-4972-837c-6884a277d3da None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:10.499211 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:10.505355 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d4a89-ab78-4972-837c-6884a277d3da None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:10.544325 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d435b732-6c38-4f1c-a84a-8848432d58ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:10.544999 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435b732-6c38-4f1c-a84a-8848432d58ff None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:10.545283 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435b732-6c38-4f1c-a84a-8848432d58ff None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:10.545652 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435b732-6c38-4f1c-a84a-8848432d58ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:10.552841 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d435b732-6c38-4f1c-a84a-8848432d58ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:10.554281 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d435b732-6c38-4f1c-a84a-8848432d58ff None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:10.566243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-d435b732-6c38-4f1c-a84a-8848432d58ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:07:10.566243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:07:10.566243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:07:10.566243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:07:10.566243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:07:10.566243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:07:10.566243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:07:10.566243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:07:10.566243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:07:10.566243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:07:10.566243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:07:10.566243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:07:10.566243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:07:10.566243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:07:10.566243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:07:10.567468 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:07:10.567468 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:07:10.567468 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:07:10.567468 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:07:10.567468 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:07:10.567468 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:07:10.567468 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:07:10.567468 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:07:10.567468 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:07:10.567468 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:07:10.567468 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:07:10.567468 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:07:10.567468 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:07:10.567468 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:07:10.567468 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:07:10.567468 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:07:10.567468 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:07:10.568707 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:07:10.568707 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:07:10.568707 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:07:10.568707 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:07:10.568707 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:07:10.568707 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:07:10.568707 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:07:10.568707 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:07:10.568707 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 537/1082] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:07:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:07:10.573902 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3196187e-ddd2-487e-9be6-79fd44b03def None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:10.579082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:10.687949 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3196187e-ddd2-487e-9be6-79fd44b03def None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:10.725939 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eaeea1-8ccd-4499-9bdb-95ae94215466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:10.726581 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eaeea1-8ccd-4499-9bdb-95ae94215466 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:10.726861 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eaeea1-8ccd-4499-9bdb-95ae94215466 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:10.727134 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eaeea1-8ccd-4499-9bdb-95ae94215466 None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:10.727898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eaeea1-8ccd-4499-9bdb-95ae94215466 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:10.728231 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eaeea1-8ccd-4499-9bdb-95ae94215466 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:10.729768 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eaeea1-8ccd-4499-9bdb-95ae94215466 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:10.743540 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 546/1083] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:07:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:10.750620 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d435b732-6c38-4f1c-a84a-8848432d58ff None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:10.784863 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74953f60-d85a-4cba-8d62-7a8c913d06a0 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:10.785147 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74953f60-d85a-4cba-8d62-7a8c913d06a0 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:10.785424 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74953f60-d85a-4cba-8d62-7a8c913d06a0 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:10.786775 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 547/1084] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:07:10.792330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9edf4-392b-497c-be43-ec5d37e1a2af None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:10.792609 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9edf4-392b-497c-be43-ec5d37e1a2af None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:10.792884 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9edf4-392b-497c-be43-ec5d37e1a2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:10.811780 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-8cf9edf4-392b-497c-be43-ec5d37e1a2af None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:07:10.812976 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8cf9edf4-392b-497c-be43-ec5d37e1a2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:07:10.875903 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:10.879861 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d435b732-6c38-4f1c-a84a-8848432d58ff None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:10.884533 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 548/1085] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:07:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:07:10.917450 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8434d-f9ea-41b6-a5cc-58ac5cbb53e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:10.918112 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8434d-f9ea-41b6-a5cc-58ac5cbb53e5 None admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:10.918390 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8434d-f9ea-41b6-a5cc-58ac5cbb53e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:10.918666 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8434d-f9ea-41b6-a5cc-58ac5cbb53e5 None admin] PATH_INFO: `/v3/projects/fa3178684d184ec880f17640d19d55b7/users/7cda290418504f0db85d665479a05353/roles/e4660ff7081a4aaa8e0d840b2e6c6dcd` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:10.921336 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e8434d-f9ea-41b6-a5cc-58ac5cbb53e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=fa3178684d184ec880f17640d19d55b7, user_id=7cda290418504f0db85d665479a05353, role_id=e4660ff7081a4aaa8e0d840b2e6c6dcd)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:10.922926 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e8434d-f9ea-41b6-a5cc-58ac5cbb53e5 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:10.937997 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 538/1086] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Sep 5 12:07:10 2022] PUT /identity/v3/projects/fa3178684d184ec880f17640d19d55b7/users/7cda290418504f0db85d665479a05353/roles/e4660ff7081a4aaa8e0d840b2e6c6dcd => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:07:11.159241 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9edf4-392b-497c-be43-ec5d37e1a2af None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:11.281194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9edf4-392b-497c-be43-ec5d37e1a2af None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:11.320466 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934804fa-5fb1-4182-8a33-38017361194c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:11.321119 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934804fa-5fb1-4182-8a33-38017361194c None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:11.321392 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934804fa-5fb1-4182-8a33-38017361194c None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:11.321657 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934804fa-5fb1-4182-8a33-38017361194c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:11.333134 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934804fa-5fb1-4182-8a33-38017361194c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:07:11.337894 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-934804fa-5fb1-4182-8a33-38017361194c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:07:11.337894 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:07:11.337894 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:07:11.337894 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:07:11.337894 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:07:11.337894 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:07:11.337894 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:07:11.337894 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:07:11.337894 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:07:11.337894 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:07:11.337894 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:07:11.337894 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:07:11.337894 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:07:11.337894 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:07:11.337894 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:07:11.337894 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:07:11.338905 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:07:11.338905 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:07:11.338905 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:07:11.338905 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:07:11.338905 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:07:11.338905 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:07:11.338905 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:07:11.338905 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:07:11.338905 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:07:11.338905 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:07:11.338905 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:07:11.338905 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:07:11.338905 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:07:11.338905 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:07:11.338905 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:07:11.338905 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:07:11.340056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:07:11.340056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:07:11.340056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:07:11.340056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:07:11.340056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:07:11.340056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:07:11.340056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:07:11.340056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:07:11.340056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:07:11.340056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:07:11.340056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:07:11.340056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:07:11.340056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 549/1087] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:07:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:07:11.345422 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8434d-f9ea-41b6-a5cc-58ac5cbb53e5 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:11.365261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:11.458471 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8434d-f9ea-41b6-a5cc-58ac5cbb53e5 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:11.495504 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de07e99-9e15-4196-aeba-d135ab435ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:11.496153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de07e99-9e15-4196-aeba-d135ab435ef1 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:11.496487 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de07e99-9e15-4196-aeba-d135ab435ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:11.497254 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de07e99-9e15-4196-aeba-d135ab435ef1 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:11.497925 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de07e99-9e15-4196-aeba-d135ab435ef1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:11.498227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de07e99-9e15-4196-aeba-d135ab435ef1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:11.499634 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de07e99-9e15-4196-aeba-d135ab435ef1 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:11.509469 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 539/1088] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:07:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:11.515219 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934804fa-5fb1-4182-8a33-38017361194c None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:11.589535 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:11.639398 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934804fa-5fb1-4182-8a33-38017361194c None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:11.685369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221aca1e-c631-449a-95af-1060eb0be644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:11.686014 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221aca1e-c631-449a-95af-1060eb0be644 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:11.686299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221aca1e-c631-449a-95af-1060eb0be644 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:11.686688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221aca1e-c631-449a-95af-1060eb0be644 None admin] PATH_INFO: `/v3/users/9a8cf6fc9d9e43b8a4a63f8615d4684d` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:11.688332 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221aca1e-c631-449a-95af-1060eb0be644 None admin] RBAC: Authorizing `identity:get_user(user_id=9a8cf6fc9d9e43b8a4a63f8615d4684d)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:11.689798 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221aca1e-c631-449a-95af-1060eb0be644 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:11.691441 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 550/1089] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:07:11 2022] GET /identity/v3/users/9a8cf6fc9d9e43b8a4a63f8615d4684d => generated 285 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:11.697643 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de07e99-9e15-4196-aeba-d135ab435ef1 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:11.794505 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:11.820485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de07e99-9e15-4196-aeba-d135ab435ef1 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:11.860392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7509a5a-0eeb-415a-8fe4-a4fe8a2955a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:11.860997 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7509a5a-0eeb-415a-8fe4-a4fe8a2955a3 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:11.861278 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7509a5a-0eeb-415a-8fe4-a4fe8a2955a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:11.861554 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7509a5a-0eeb-415a-8fe4-a4fe8a2955a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:11.936873 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7509a5a-0eeb-415a-8fe4-a4fe8a2955a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:11.938507 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7509a5a-0eeb-415a-8fe4-a4fe8a2955a3 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:12.027190 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:12.055179 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 540/1090] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:07:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:07:12.061253 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221aca1e-c631-449a-95af-1060eb0be644 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:12.188779 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221aca1e-c631-449a-95af-1060eb0be644 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:12.214129 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:12.229130 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e24c7-299e-4010-9f28-d48d4b88eaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:12.229809 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e24c7-299e-4010-9f28-d48d4b88eaa6 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:12.230097 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e24c7-299e-4010-9f28-d48d4b88eaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:12.230433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e24c7-299e-4010-9f28-d48d4b88eaa6 None admin] PATH_INFO: `/v3/projects/c6cb339307b14a88b35ef07cc196e492` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:12.231680 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146e24c7-299e-4010-9f28-d48d4b88eaa6 None admin] RBAC: Authorizing `identity:get_project(project_id=c6cb339307b14a88b35ef07cc196e492)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:12.233121 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146e24c7-299e-4010-9f28-d48d4b88eaa6 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:12.234806 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 551/1091] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:07:12 2022] GET /identity/v3/projects/c6cb339307b14a88b35ef07cc196e492 => generated 311 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:12.242150 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7509a5a-0eeb-415a-8fe4-a4fe8a2955a3 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:12.365381 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7509a5a-0eeb-415a-8fe4-a4fe8a2955a3 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:12.407858 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13cd8a-6bf2-446a-9d62-32736e59738c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:12.408483 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13cd8a-6bf2-446a-9d62-32736e59738c None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:12.408852 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13cd8a-6bf2-446a-9d62-32736e59738c None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:12.409204 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13cd8a-6bf2-446a-9d62-32736e59738c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:12.410005 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a13cd8a-6bf2-446a-9d62-32736e59738c None admin] RBAC: Adding query filter params (role.id=618c48706f7f445abeddf7eb2f8e927a, scope.project.id=c6cb339307b14a88b35ef07cc196e492, user.id=9a8cf6fc9d9e43b8a4a63f8615d4684d) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:12.410388 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a13cd8a-6bf2-446a-9d62-32736e59738c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:12.411952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a13cd8a-6bf2-446a-9d62-32736e59738c None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:12.419620 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 541/1092] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:07:12 2022] GET /identity/v3/role_assignments?user.id=9a8cf6fc9d9e43b8a4a63f8615d4684d&scope.project.id=c6cb339307b14a88b35ef07cc196e492&role.id=618c48706f7f445abeddf7eb2f8e927a => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:12.956932 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39666d2a-547d-419d-b6b0-a42a6d5da0b5 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:12.957549 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39666d2a-547d-419d-b6b0-a42a6d5da0b5 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:12.957549 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39666d2a-547d-419d-b6b0-a42a6d5da0b5 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:12.958809 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 552/1093] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:07:12.964286 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c60df5-7dfe-4f7e-b40f-9feb5acc27ff None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:12.964785 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c60df5-7dfe-4f7e-b40f-9feb5acc27ff None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:12.964889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c60df5-7dfe-4f7e-b40f-9feb5acc27ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:12.984878 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-62c60df5-7dfe-4f7e-b40f-9feb5acc27ff None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:07:12.986141 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-62c60df5-7dfe-4f7e-b40f-9feb5acc27ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:07:13.023723 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:13.067994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 542/1094] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:07:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:07:13.365997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39666d2a-547d-419d-b6b0-a42a6d5da0b5 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:13.487237 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39666d2a-547d-419d-b6b0-a42a6d5da0b5 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:13.526192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2a76a-e341-45f7-b2ed-553ab65ffd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:13.526851 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2a76a-e341-45f7-b2ed-553ab65ffd8b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:13.527174 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2a76a-e341-45f7-b2ed-553ab65ffd8b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:13.527516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2a76a-e341-45f7-b2ed-553ab65ffd8b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:13.545678 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2a76a-e341-45f7-b2ed-553ab65ffd8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:07:13.550596 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-cac2a76a-e341-45f7-b2ed-553ab65ffd8b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 12:07:13.550596 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:07:13.550596 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:07:13.550596 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:07:13.550596 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:07:13.550596 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:07:13.550596 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:07:13.550596 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:07:13.550596 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:07:13.550596 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:07:13.550596 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:07:13.550596 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:07:13.550596 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:07:13.550596 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:07:13.550596 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:07:13.550596 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:07:13.555887 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:07:13.555887 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:07:13.555887 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:07:13.555887 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:07:13.555887 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:07:13.555887 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:07:13.555887 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:07:13.555887 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:07:13.555887 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:07:13.555887 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:07:13.555887 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:07:13.555887 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:07:13.555887 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:07:13.555887 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:07:13.555887 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:07:13.555887 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:07:13.557210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:07:13.557210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:07:13.557210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:07:13.557210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:07:13.557210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:07:13.557210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:07:13.557210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:07:13.557210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:07:13.557210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:07:13.557210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:07:13.557210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 12:07:13.557210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:07:13.557210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 553/1095] 10.222.0.16 () {56 vars in 1120 bytes} [Mon Sep 5 12:07:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:07:13.558292 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c60df5-7dfe-4f7e-b40f-9feb5acc27ff None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:13.676826 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c60df5-7dfe-4f7e-b40f-9feb5acc27ff None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:13.714583 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac53a78-34ab-40a8-a8ab-85208334b973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:13.715924 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac53a78-34ab-40a8-a8ab-85208334b973 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:13.716309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac53a78-34ab-40a8-a8ab-85208334b973 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:13.716602 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac53a78-34ab-40a8-a8ab-85208334b973 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:13.717329 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac53a78-34ab-40a8-a8ab-85208334b973 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:13.717655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac53a78-34ab-40a8-a8ab-85208334b973 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:13.719168 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac53a78-34ab-40a8-a8ab-85208334b973 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:13.728858 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 543/1096] 10.222.0.16 () {56 vars in 1115 bytes} [Mon Sep 5 12:07:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:13.734666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2a76a-e341-45f7-b2ed-553ab65ffd8b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:13.792253 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:13.847236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2a76a-e341-45f7-b2ed-553ab65ffd8b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:13.883100 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b1ea4-4c69-43c1-ba68-4dbd3876a956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:13.883776 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b1ea4-4c69-43c1-ba68-4dbd3876a956 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:13.884055 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b1ea4-4c69-43c1-ba68-4dbd3876a956 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:13.884325 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b1ea4-4c69-43c1-ba68-4dbd3876a956 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:13.896367 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336b1ea4-4c69-43c1-ba68-4dbd3876a956 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:13.897851 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336b1ea4-4c69-43c1-ba68-4dbd3876a956 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:13.914770 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-336b1ea4-4c69-43c1-ba68-4dbd3876a956 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:07:13.914770 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:07:13.914770 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:07:13.914770 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:07:13.914770 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:07:13.914770 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:07:13.914770 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:07:13.914770 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:07:13.914770 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:07:13.914770 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:07:13.914770 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:07:13.914770 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:07:13.914770 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:07:13.914770 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:07:13.914770 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:07:13.914770 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:07:13.915946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:07:13.915946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:07:13.915946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:07:13.915946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:07:13.915946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:07:13.915946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:07:13.915946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:07:13.915946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:07:13.915946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:07:13.915946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:07:13.915946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:07:13.915946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:07:13.915946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:07:13.915946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:07:13.915946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:07:13.915946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:07:13.915946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:07:13.917514 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:07:13.917514 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:07:13.917514 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:07:13.917514 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 554/1097] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:07:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:07:13.921971 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac53a78-34ab-40a8-a8ab-85208334b973 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:13.960400 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:14.037361 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac53a78-34ab-40a8-a8ab-85208334b973 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:14.080676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8260de3-538d-4b21-a215-2ebf82d0d8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:14.081410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8260de3-538d-4b21-a215-2ebf82d0d8a4 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:14.081789 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8260de3-538d-4b21-a215-2ebf82d0d8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:14.082153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8260de3-538d-4b21-a215-2ebf82d0d8a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:14.082897 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8260de3-538d-4b21-a215-2ebf82d0d8a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:14.083297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8260de3-538d-4b21-a215-2ebf82d0d8a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:14.085124 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8260de3-538d-4b21-a215-2ebf82d0d8a4 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:14.101552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 544/1098] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:07:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:14.108010 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b1ea4-4c69-43c1-ba68-4dbd3876a956 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:14.224316 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b1ea4-4c69-43c1-ba68-4dbd3876a956 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:14.226017 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:14.269795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f414941-a9e8-4941-afbe-7cc6b1bcf539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:14.270497 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f414941-a9e8-4941-afbe-7cc6b1bcf539 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:14.270770 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f414941-a9e8-4941-afbe-7cc6b1bcf539 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:14.271121 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f414941-a9e8-4941-afbe-7cc6b1bcf539 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:14.279876 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f414941-a9e8-4941-afbe-7cc6b1bcf539 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:14.281450 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f414941-a9e8-4941-afbe-7cc6b1bcf539 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:14.295448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-1f414941-a9e8-4941-afbe-7cc6b1bcf539 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:07:14.295448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:07:14.295448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:07:14.295448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:07:14.295448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:07:14.295448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:07:14.295448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:07:14.295448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:07:14.295448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:07:14.295448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:07:14.295448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:07:14.295448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:07:14.295448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:07:14.295448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:07:14.295448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:07:14.295448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:07:14.296587 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:07:14.296587 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:07:14.296587 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:07:14.296587 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:07:14.296587 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:07:14.296587 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:07:14.296587 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:07:14.296587 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:07:14.296587 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:07:14.296587 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:07:14.296587 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:07:14.296587 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:07:14.296587 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:07:14.296587 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:07:14.296587 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:07:14.296587 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:07:14.296587 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:07:14.296587 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:07:14.297671 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:07:14.297671 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:07:14.297671 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:07:14.297671 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:07:14.297671 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:07:14.297671 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:07:14.297671 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:07:14.297671 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:07:14.297671 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:07:14.297671 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:07:14.297671 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 555/1099] 10.222.0.16 () {56 vars in 1093 bytes} [Mon Sep 5 12:07:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:07:14.302983 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8260de3-538d-4b21-a215-2ebf82d0d8a4 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:14.422516 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:14.430358 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8260de3-538d-4b21-a215-2ebf82d0d8a4 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:14.467879 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39764c-52e1-4e8e-bff5-8facc003d48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:14.468482 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39764c-52e1-4e8e-bff5-8facc003d48e None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:14.468759 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39764c-52e1-4e8e-bff5-8facc003d48e None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:14.469028 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39764c-52e1-4e8e-bff5-8facc003d48e None admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:14.469774 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39764c-52e1-4e8e-bff5-8facc003d48e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:14.470080 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39764c-52e1-4e8e-bff5-8facc003d48e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:14.471497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39764c-52e1-4e8e-bff5-8facc003d48e None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:14.487469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94561b0-f57d-43e1-8dde-cb4a3780743e None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:14.487914 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94561b0-f57d-43e1-8dde-cb4a3780743e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:14.488391 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94561b0-f57d-43e1-8dde-cb4a3780743e None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:14.490917 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 556/1100] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:07:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:07:14.491089 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 545/1101] 10.222.0.16 () {56 vars in 1133 bytes} [Mon Sep 5 12:07:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:14.497002 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd80510-d1f1-4023-8215-f4ac04625c20 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:14.497299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd80510-d1f1-4023-8215-f4ac04625c20 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:14.497515 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39764c-52e1-4e8e-bff5-8facc003d48e None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:14.497628 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd80510-d1f1-4023-8215-f4ac04625c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:14.517292 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-abd80510-d1f1-4023-8215-f4ac04625c20 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:07:14.518640 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-abd80510-d1f1-4023-8215-f4ac04625c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:07:14.602690 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 557/1102] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:07:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:07:14.613060 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39764c-52e1-4e8e-bff5-8facc003d48e None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:14.658052 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d5a0b-2d5b-4934-a2fc-561f168cbc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:14.658789 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d5a0b-2d5b-4934-a2fc-561f168cbc7a None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:14.659079 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d5a0b-2d5b-4934-a2fc-561f168cbc7a None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:14.659493 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d5a0b-2d5b-4934-a2fc-561f168cbc7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:14.682073 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:14.739093 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d5a0b-2d5b-4934-a2fc-561f168cbc7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:14.740552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d5a0b-2d5b-4934-a2fc-561f168cbc7a None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:14.859001 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 546/1103] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:07:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 05 12:07:14.865642 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd80510-d1f1-4023-8215-f4ac04625c20 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:14.884451 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d5a0b-2d5b-4934-a2fc-561f168cbc7a None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:14.953227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:14.975782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:14.987289 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd80510-d1f1-4023-8215-f4ac04625c20 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:15.010350 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d5a0b-2d5b-4934-a2fc-561f168cbc7a None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:15.024297 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc55430-4ca7-4995-9e3a-68f3de9e3b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:15.025022 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc55430-4ca7-4995-9e3a-68f3de9e3b64 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:15.025377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc55430-4ca7-4995-9e3a-68f3de9e3b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:15.025659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc55430-4ca7-4995-9e3a-68f3de9e3b64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:15.038088 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc55430-4ca7-4995-9e3a-68f3de9e3b64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:15.039617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc55430-4ca7-4995-9e3a-68f3de9e3b64 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:15.054015 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ce0c1-c45f-4fb9-b6f0-dda63269dfbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:15.054772 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ce0c1-c45f-4fb9-b6f0-dda63269dfbc None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:15.055057 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ce0c1-c45f-4fb9-b6f0-dda63269dfbc None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:15.055456 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ce0c1-c45f-4fb9-b6f0-dda63269dfbc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:15.056968 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ce0c1-c45f-4fb9-b6f0-dda63269dfbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:15.057590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-4fc55430-4ca7-4995-9e3a-68f3de9e3b64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:07:15.057590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:07:15.057590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:07:15.057590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:07:15.057590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:07:15.057590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:07:15.057590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:07:15.057590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:07:15.057590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:07:15.057590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:07:15.057590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:07:15.057590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:07:15.057590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:07:15.057590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:07:15.057590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:07:15.057590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:07:15.058795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:07:15.058795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:07:15.058795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:07:15.058795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:07:15.058795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:07:15.058795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:07:15.058795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:07:15.058795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:07:15.058795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:07:15.058795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:07:15.058795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:07:15.058795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:07:15.058795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:07:15.058795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:07:15.058795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:07:15.058795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:07:15.058795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:07:15.059880 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:07:15.059880 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:07:15.059880 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:07:15.059880 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 558/1104] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:07:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:07:15.060113 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ce0c1-c45f-4fb9-b6f0-dda63269dfbc None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:15.063635 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 547/1105] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:07:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:07:15.064695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc55430-4ca7-4995-9e3a-68f3de9e3b64 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:15.067470 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ce0c1-c45f-4fb9-b6f0-dda63269dfbc None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:15.183061 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ce0c1-c45f-4fb9-b6f0-dda63269dfbc None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:15.206598 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:15.207261 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc55430-4ca7-4995-9e3a-68f3de9e3b64 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:15.223076 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b847ae9-822b-438a-8283-abe699a8d75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:15.223719 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b847ae9-822b-438a-8283-abe699a8d75f None admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:15.224019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b847ae9-822b-438a-8283-abe699a8d75f None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:15.224286 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b847ae9-822b-438a-8283-abe699a8d75f None admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:15.224939 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b847ae9-822b-438a-8283-abe699a8d75f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:15.226357 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b847ae9-822b-438a-8283-abe699a8d75f None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:15.250583 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 548/1106] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Sep 5 12:07:15 2022] POST /identity/v3/users => generated 286 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:07:15.252025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e3fdf-7995-41fa-b5e0-216ae8a1ec5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:15.252691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e3fdf-7995-41fa-b5e0-216ae8a1ec5a None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:15.252985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e3fdf-7995-41fa-b5e0-216ae8a1ec5a None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:15.253260 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e3fdf-7995-41fa-b5e0-216ae8a1ec5a None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:15.253934 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e3fdf-7995-41fa-b5e0-216ae8a1ec5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:15.254298 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e3fdf-7995-41fa-b5e0-216ae8a1ec5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:15.255793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e3fdf-7995-41fa-b5e0-216ae8a1ec5a None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:15.271718 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 559/1107] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:15.281785 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b847ae9-822b-438a-8283-abe699a8d75f None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:15.412059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b847ae9-822b-438a-8283-abe699a8d75f None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:15.451381 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4747e-12b5-4c41-84f8-b4219f03ca8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:15.452210 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4747e-12b5-4c41-84f8-b4219f03ca8d None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:15.452499 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4747e-12b5-4c41-84f8-b4219f03ca8d None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:15.452773 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4747e-12b5-4c41-84f8-b4219f03ca8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:15.461107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d4747e-12b5-4c41-84f8-b4219f03ca8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:15.462680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d4747e-12b5-4c41-84f8-b4219f03ca8d None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:15.476554 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-d7d4747e-12b5-4c41-84f8-b4219f03ca8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:07:15.476554 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:07:15.476554 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:07:15.476554 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:07:15.476554 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:07:15.476554 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:07:15.476554 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:07:15.476554 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:07:15.476554 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:07:15.476554 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:07:15.476554 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:07:15.476554 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:07:15.476554 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:07:15.476554 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:07:15.476554 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:07:15.478604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:07:15.478604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:07:15.478604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:07:15.478604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:07:15.478604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:07:15.478604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:07:15.478604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:07:15.478604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:07:15.478604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:07:15.478604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:07:15.478604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:07:15.478604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:07:15.478604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:07:15.478604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:07:15.478604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:07:15.478604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:07:15.478604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:07:15.480780 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:07:15.480780 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:07:15.480780 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:07:15.480780 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:07:15.480780 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:07:15.480780 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:07:15.480780 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:07:15.480780 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:07:15.480780 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 549/1108] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:07:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:07:15.485024 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e3fdf-7995-41fa-b5e0-216ae8a1ec5a None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:15.489298 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:15.599933 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e3fdf-7995-41fa-b5e0-216ae8a1ec5a None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:15.636916 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d7d8a-f8a6-4355-9291-e348a0416479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:15.638087 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d7d8a-f8a6-4355-9291-e348a0416479 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:15.638364 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d7d8a-f8a6-4355-9291-e348a0416479 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:15.638638 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d7d8a-f8a6-4355-9291-e348a0416479 None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:15.639364 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d7d8a-f8a6-4355-9291-e348a0416479 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:15.639701 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d7d8a-f8a6-4355-9291-e348a0416479 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:15.641157 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d7d8a-f8a6-4355-9291-e348a0416479 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:15.654449 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 560/1109] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:07:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:07:15.660770 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4747e-12b5-4c41-84f8-b4219f03ca8d None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:15.706663 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:15.797795 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4747e-12b5-4c41-84f8-b4219f03ca8d None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:15.853313 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f4a732-6ec0-4344-bca8-c8765e343424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:15.854267 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4a732-6ec0-4344-bca8-c8765e343424 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:15.854759 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4a732-6ec0-4344-bca8-c8765e343424 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:15.855233 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4a732-6ec0-4344-bca8-c8765e343424 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:15.856696 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f4a732-6ec0-4344-bca8-c8765e343424 None admin] RBAC: Adding query filter params (role.id=e4660ff7081a4aaa8e0d840b2e6c6dcd, scope.project.id=fa3178684d184ec880f17640d19d55b7, user.id=7cda290418504f0db85d665479a05353) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:15.857175 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f4a732-6ec0-4344-bca8-c8765e343424 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:15.859787 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f4a732-6ec0-4344-bca8-c8765e343424 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:15.879565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 550/1110] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:07:15 2022] GET /identity/v3/role_assignments?user.id=7cda290418504f0db85d665479a05353&scope.project.id=fa3178684d184ec880f17640d19d55b7&role.id=e4660ff7081a4aaa8e0d840b2e6c6dcd => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:07:17.407106 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93d797-3d1e-4c4d-8acf-af92dfe5a338 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:17.407696 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93d797-3d1e-4c4d-8acf-af92dfe5a338 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:17.407998 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93d797-3d1e-4c4d-8acf-af92dfe5a338 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:17.409570 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 561/1111] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:07:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:07:17.416332 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae67f04-cc36-4c06-b391-72ad1a252364 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:17.416814 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae67f04-cc36-4c06-b391-72ad1a252364 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:17.416906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae67f04-cc36-4c06-b391-72ad1a252364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:17.437086 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-bae67f04-cc36-4c06-b391-72ad1a252364 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:07:17.438299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-bae67f04-cc36-4c06-b391-72ad1a252364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:07:17.511015 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 551/1112] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:07:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:07:17.801923 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93d797-3d1e-4c4d-8acf-af92dfe5a338 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:17.871258 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:17.915417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93d797-3d1e-4c4d-8acf-af92dfe5a338 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:17.950611 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f06695-c69f-4e60-bf2f-119f66032064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:17.951287 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f06695-c69f-4e60-bf2f-119f66032064 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:17.951579 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f06695-c69f-4e60-bf2f-119f66032064 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:17.951858 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f06695-c69f-4e60-bf2f-119f66032064 None admin] PATH_INFO: `/v3/roles/20bf8c77837d43b590a2bd9a67d01f84` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:17.953252 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f06695-c69f-4e60-bf2f-119f66032064 None admin] RBAC: Authorizing `identity:get_role(role_id=20bf8c77837d43b590a2bd9a67d01f84)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:17.954609 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f06695-c69f-4e60-bf2f-119f66032064 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:17.955985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 562/1113] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:07:17 2022] GET /identity/v3/roles/20bf8c77837d43b590a2bd9a67d01f84 => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:17.964736 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae67f04-cc36-4c06-b391-72ad1a252364 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:18.001622 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:18.080319 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae67f04-cc36-4c06-b391-72ad1a252364 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:18.117410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ef0ca-fa19-4d8f-99a0-03ed0eab08fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:18.118031 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ef0ca-fa19-4d8f-99a0-03ed0eab08fd None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:18.118307 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ef0ca-fa19-4d8f-99a0-03ed0eab08fd None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:18.118572 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ef0ca-fa19-4d8f-99a0-03ed0eab08fd None admin] PATH_INFO: `/v3/users/1993d4952e4540d498add8ccc9416996` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:18.140482 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ef0ca-fa19-4d8f-99a0-03ed0eab08fd None admin] RBAC: Authorizing `identity:get_user(user_id=1993d4952e4540d498add8ccc9416996)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:18.142801 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ef0ca-fa19-4d8f-99a0-03ed0eab08fd None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:18.145850 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 552/1114] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:07:17 2022] GET /identity/v3/users/1993d4952e4540d498add8ccc9416996 => generated 286 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:18.152250 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f06695-c69f-4e60-bf2f-119f66032064 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:18.260081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f06695-c69f-4e60-bf2f-119f66032064 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:18.294565 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:18.300551 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f356ba-2ba0-40b3-8984-fe4452bbacb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:18.301168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f356ba-2ba0-40b3-8984-fe4452bbacb5 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:18.301448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f356ba-2ba0-40b3-8984-fe4452bbacb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:18.301716 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f356ba-2ba0-40b3-8984-fe4452bbacb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:18.377018 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f356ba-2ba0-40b3-8984-fe4452bbacb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:18.378490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f356ba-2ba0-40b3-8984-fe4452bbacb5 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:18.499673 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 563/1115] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:07:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:07:18.506002 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ef0ca-fa19-4d8f-99a0-03ed0eab08fd None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:18.617413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:18.635710 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ef0ca-fa19-4d8f-99a0-03ed0eab08fd None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:18.695360 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cadcf7-5d09-445f-bc95-6fd021b51415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:18.696494 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cadcf7-5d09-445f-bc95-6fd021b51415 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:18.696794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cadcf7-5d09-445f-bc95-6fd021b51415 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:18.697209 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cadcf7-5d09-445f-bc95-6fd021b51415 None admin] PATH_INFO: `/v3/projects/c6cb339307b14a88b35ef07cc196e492` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:18.698825 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cadcf7-5d09-445f-bc95-6fd021b51415 None admin] RBAC: Authorizing `identity:get_project(project_id=c6cb339307b14a88b35ef07cc196e492)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:18.702135 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cadcf7-5d09-445f-bc95-6fd021b51415 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:18.705025 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 553/1116] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:07:18 2022] GET /identity/v3/projects/c6cb339307b14a88b35ef07cc196e492 => generated 311 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:18.711253 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f356ba-2ba0-40b3-8984-fe4452bbacb5 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:18.744929 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:18.819081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f356ba-2ba0-40b3-8984-fe4452bbacb5 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:18.856750 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9423c15-10e7-4eb8-9ef3-afefe045fcf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:18.857365 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9423c15-10e7-4eb8-9ef3-afefe045fcf4 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:18.857641 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9423c15-10e7-4eb8-9ef3-afefe045fcf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:18.857910 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9423c15-10e7-4eb8-9ef3-afefe045fcf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:18.858646 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9423c15-10e7-4eb8-9ef3-afefe045fcf4 None admin] RBAC: Adding query filter params (role.id=20bf8c77837d43b590a2bd9a67d01f84, scope.project.id=c6cb339307b14a88b35ef07cc196e492, user.id=1993d4952e4540d498add8ccc9416996) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:18.858953 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9423c15-10e7-4eb8-9ef3-afefe045fcf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:18.860458 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9423c15-10e7-4eb8-9ef3-afefe045fcf4 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:18.867992 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 564/1117] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:07:18 2022] GET /identity/v3/role_assignments?user.id=1993d4952e4540d498add8ccc9416996&scope.project.id=c6cb339307b14a88b35ef07cc196e492&role.id=20bf8c77837d43b590a2bd9a67d01f84 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:20.790646 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f10254-d2e5-44f2-8001-ffa609e09d09 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:20.791253 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f10254-d2e5-44f2-8001-ffa609e09d09 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:20.791253 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f10254-d2e5-44f2-8001-ffa609e09d09 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:20.792588 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 554/1118] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:07:20.798677 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e868b-98b3-4f12-80ca-2a4778acb240 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:20.799126 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e868b-98b3-4f12-80ca-2a4778acb240 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:20.799262 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e868b-98b3-4f12-80ca-2a4778acb240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:20.817234 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-a16e868b-98b3-4f12-80ca-2a4778acb240 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:07:20.818428 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a16e868b-98b3-4f12-80ca-2a4778acb240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:07:20.884583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 565/1119] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:07:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:07:21.159495 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f10254-d2e5-44f2-8001-ffa609e09d09 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:21.271245 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f10254-d2e5-44f2-8001-ffa609e09d09 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:21.305413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8073061-566d-4f33-a810-301b7395402c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:21.306037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8073061-566d-4f33-a810-301b7395402c None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:21.306311 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8073061-566d-4f33-a810-301b7395402c None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:21.306579 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8073061-566d-4f33-a810-301b7395402c None admin] PATH_INFO: `/v3/roles/20bf8c77837d43b590a2bd9a67d01f84` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:21.307967 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8073061-566d-4f33-a810-301b7395402c None admin] RBAC: Authorizing `identity:get_role(role_id=20bf8c77837d43b590a2bd9a67d01f84)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:21.309349 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8073061-566d-4f33-a810-301b7395402c None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:21.310645 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 555/1120] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:07:21 2022] GET /identity/v3/roles/20bf8c77837d43b590a2bd9a67d01f84 => generated 223 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:21.316551 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e868b-98b3-4f12-80ca-2a4778acb240 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:21.348628 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:21.428622 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e868b-98b3-4f12-80ca-2a4778acb240 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:21.462423 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ebba5-79c1-4495-be5b-d32ecd4ce9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:21.463077 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ebba5-79c1-4495-be5b-d32ecd4ce9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:21.463357 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ebba5-79c1-4495-be5b-d32ecd4ce9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:21.463660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ebba5-79c1-4495-be5b-d32ecd4ce9d5 None admin] PATH_INFO: `/v3/users/1993d4952e4540d498add8ccc9416996` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:21.464785 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ebba5-79c1-4495-be5b-d32ecd4ce9d5 None admin] RBAC: Authorizing `identity:get_user(user_id=1993d4952e4540d498add8ccc9416996)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:21.466792 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ebba5-79c1-4495-be5b-d32ecd4ce9d5 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:21.468431 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 566/1121] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:07:21 2022] GET /identity/v3/users/1993d4952e4540d498add8ccc9416996 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:21.474560 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8073061-566d-4f33-a810-301b7395402c None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:21.499705 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:21.576455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8073061-566d-4f33-a810-301b7395402c None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:21.610001 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b27d60-4bc1-4f50-bf88-139f28f85326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:21.610624 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b27d60-4bc1-4f50-bf88-139f28f85326 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:21.610903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b27d60-4bc1-4f50-bf88-139f28f85326 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:21.611172 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b27d60-4bc1-4f50-bf88-139f28f85326 None admin] PATH_INFO: `/v3/projects/c6cb339307b14a88b35ef07cc196e492` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:21.612311 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b27d60-4bc1-4f50-bf88-139f28f85326 None admin] RBAC: Authorizing `identity:get_project(project_id=c6cb339307b14a88b35ef07cc196e492)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:21.613733 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b27d60-4bc1-4f50-bf88-139f28f85326 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:21.615339 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 556/1122] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:07:21 2022] GET /identity/v3/projects/c6cb339307b14a88b35ef07cc196e492 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:21.621947 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ebba5-79c1-4495-be5b-d32ecd4ce9d5 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:21.722711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:21.729477 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ebba5-79c1-4495-be5b-d32ecd4ce9d5 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:21.762783 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d24ae17-d812-4cec-a527-a3af8e8519fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:21.763434 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24ae17-d812-4cec-a527-a3af8e8519fc None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:21.763732 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24ae17-d812-4cec-a527-a3af8e8519fc None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:21.763998 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24ae17-d812-4cec-a527-a3af8e8519fc None admin] PATH_INFO: `/v3/projects/c6cb339307b14a88b35ef07cc196e492/users/1993d4952e4540d498add8ccc9416996/roles/20bf8c77837d43b590a2bd9a67d01f84` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:21.766101 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d24ae17-d812-4cec-a527-a3af8e8519fc None admin] RBAC: Authorizing `identity:create_grant(project_id=c6cb339307b14a88b35ef07cc196e492, user_id=1993d4952e4540d498add8ccc9416996, role_id=20bf8c77837d43b590a2bd9a67d01f84)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:21.767571 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d24ae17-d812-4cec-a527-a3af8e8519fc None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:21.781525 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 567/1123] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Sep 5 12:07:21 2022] PUT /identity/v3/projects/c6cb339307b14a88b35ef07cc196e492/users/1993d4952e4540d498add8ccc9416996/roles/20bf8c77837d43b590a2bd9a67d01f84 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:07:23.721809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03ad40-7378-4423-a77c-887b8dd90529 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:23.722333 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03ad40-7378-4423-a77c-887b8dd90529 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:23.722333 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03ad40-7378-4423-a77c-887b8dd90529 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:23.723673 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 557/1124] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:07:23.728770 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445ba39-e8f7-44a0-8a4c-88e44396357b None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:23.729193 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445ba39-e8f7-44a0-8a4c-88e44396357b None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:23.729306 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445ba39-e8f7-44a0-8a4c-88e44396357b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:23.748154 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-0445ba39-e8f7-44a0-8a4c-88e44396357b None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:07:23.749315 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-0445ba39-e8f7-44a0-8a4c-88e44396357b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:07:23.815212 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 568/1125] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:07:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:07:24.089258 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03ad40-7378-4423-a77c-887b8dd90529 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:24.185414 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:24.195577 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03ad40-7378-4423-a77c-887b8dd90529 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:24.229833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6680f21-60ff-4a1e-8549-6a35ceb41947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:24.230446 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6680f21-60ff-4a1e-8549-6a35ceb41947 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:24.230755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6680f21-60ff-4a1e-8549-6a35ceb41947 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:24.231021 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6680f21-60ff-4a1e-8549-6a35ceb41947 None admin] PATH_INFO: `/v3/roles/20bf8c77837d43b590a2bd9a67d01f84` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:24.232407 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6680f21-60ff-4a1e-8549-6a35ceb41947 None admin] RBAC: Authorizing `identity:get_role(role_id=20bf8c77837d43b590a2bd9a67d01f84)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:24.233753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6680f21-60ff-4a1e-8549-6a35ceb41947 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:24.235053 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 558/1126] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:07:24 2022] GET /identity/v3/roles/20bf8c77837d43b590a2bd9a67d01f84 => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:24.240925 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445ba39-e8f7-44a0-8a4c-88e44396357b None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:24.321507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:24.338586 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445ba39-e8f7-44a0-8a4c-88e44396357b None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:24.373402 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36ba63-1990-4e2a-b4e7-23d07c62ebe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:24.374043 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36ba63-1990-4e2a-b4e7-23d07c62ebe5 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:24.374323 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36ba63-1990-4e2a-b4e7-23d07c62ebe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:24.374598 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36ba63-1990-4e2a-b4e7-23d07c62ebe5 None admin] PATH_INFO: `/v3/users/1993d4952e4540d498add8ccc9416996` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:24.375751 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c36ba63-1990-4e2a-b4e7-23d07c62ebe5 None admin] RBAC: Authorizing `identity:get_user(user_id=1993d4952e4540d498add8ccc9416996)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:24.377185 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c36ba63-1990-4e2a-b4e7-23d07c62ebe5 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:24.378782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 569/1127] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:07:24 2022] GET /identity/v3/users/1993d4952e4540d498add8ccc9416996 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:24.384584 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6680f21-60ff-4a1e-8549-6a35ceb41947 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:24.482389 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6680f21-60ff-4a1e-8549-6a35ceb41947 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:24.515167 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fb62f-eedf-4730-9928-03a0ffa89e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:24.515789 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fb62f-eedf-4730-9928-03a0ffa89e7c None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:24.516064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fb62f-eedf-4730-9928-03a0ffa89e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:24.516332 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fb62f-eedf-4730-9928-03a0ffa89e7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:24.542960 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:24.582458 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386fb62f-eedf-4730-9928-03a0ffa89e7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:24.583923 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386fb62f-eedf-4730-9928-03a0ffa89e7c None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:24.678138 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 559/1128] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:07:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 05 12:07:24.684544 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36ba63-1990-4e2a-b4e7-23d07c62ebe5 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:24.786397 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36ba63-1990-4e2a-b4e7-23d07c62ebe5 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:24.819537 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8ef8f-0300-4aec-b207-07bf72cbe0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:24.820170 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8ef8f-0300-4aec-b207-07bf72cbe0a2 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:24.820447 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8ef8f-0300-4aec-b207-07bf72cbe0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:24.820718 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8ef8f-0300-4aec-b207-07bf72cbe0a2 None admin] PATH_INFO: `/v3/projects/c6cb339307b14a88b35ef07cc196e492` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:24.821857 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8ef8f-0300-4aec-b207-07bf72cbe0a2 None admin] RBAC: Authorizing `identity:get_project(project_id=c6cb339307b14a88b35ef07cc196e492)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:24.823291 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8ef8f-0300-4aec-b207-07bf72cbe0a2 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:24.825506 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 570/1129] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:07:24 2022] GET /identity/v3/projects/c6cb339307b14a88b35ef07cc196e492 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:24.831946 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fb62f-eedf-4730-9928-03a0ffa89e7c None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:24.908244 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:24.948966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fb62f-eedf-4730-9928-03a0ffa89e7c None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:24.983698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650cfada-9d77-4b90-a8cb-068d922c5377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:24.984301 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650cfada-9d77-4b90-a8cb-068d922c5377 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:24.984582 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650cfada-9d77-4b90-a8cb-068d922c5377 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:24.984859 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650cfada-9d77-4b90-a8cb-068d922c5377 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:24.985591 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650cfada-9d77-4b90-a8cb-068d922c5377 None admin] RBAC: Adding query filter params (role.id=20bf8c77837d43b590a2bd9a67d01f84, scope.project.id=c6cb339307b14a88b35ef07cc196e492, user.id=1993d4952e4540d498add8ccc9416996) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:24.985900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650cfada-9d77-4b90-a8cb-068d922c5377 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:24.987324 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650cfada-9d77-4b90-a8cb-068d922c5377 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:24.993151 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 560/1130] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:07:24 2022] GET /identity/v3/role_assignments?user.id=1993d4952e4540d498add8ccc9416996&scope.project.id=c6cb339307b14a88b35ef07cc196e492&role.id=20bf8c77837d43b590a2bd9a67d01f84 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:26.943761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53759b0c-e54b-4aa4-9d91-4aadfebb1647 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:26.944321 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53759b0c-e54b-4aa4-9d91-4aadfebb1647 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:26.944321 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53759b0c-e54b-4aa4-9d91-4aadfebb1647 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:26.945658 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 571/1131] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:07:26.950738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b4b0b-e2df-4c22-af75-bf5e5572fd05 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:26.951198 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b4b0b-e2df-4c22-af75-bf5e5572fd05 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:26.951294 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b4b0b-e2df-4c22-af75-bf5e5572fd05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:26.969820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-427b4b0b-e2df-4c22-af75-bf5e5572fd05 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:07:26.971013 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-427b4b0b-e2df-4c22-af75-bf5e5572fd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:07:27.036720 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 561/1132] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:07:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:07:27.308548 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53759b0c-e54b-4aa4-9d91-4aadfebb1647 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:27.330808 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:27.410673 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53759b0c-e54b-4aa4-9d91-4aadfebb1647 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:27.443744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd06b8-3add-4516-a345-94682c2f9525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:27.444369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd06b8-3add-4516-a345-94682c2f9525 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:27.444650 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd06b8-3add-4516-a345-94682c2f9525 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:27.444924 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd06b8-3add-4516-a345-94682c2f9525 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:27.446099 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cd06b8-3add-4516-a345-94682c2f9525 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:27.447550 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cd06b8-3add-4516-a345-94682c2f9525 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:27.449120 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 572/1133] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:07:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:27.455540 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b4b0b-e2df-4c22-af75-bf5e5572fd05 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:27.508462 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:27.551916 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b4b0b-e2df-4c22-af75-bf5e5572fd05 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:27.585219 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b7828-7081-4a9d-951b-59bfdede66e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:27.585749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b7828-7081-4a9d-951b-59bfdede66e4 None admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:27.586027 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b7828-7081-4a9d-951b-59bfdede66e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:27.586299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b7828-7081-4a9d-951b-59bfdede66e4 None admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:27.586941 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7b7828-7081-4a9d-951b-59bfdede66e4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:27.588383 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7b7828-7081-4a9d-951b-59bfdede66e4 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:27.613941 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 562/1134] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Sep 5 12:07:27 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:07:29.560573 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea013f14-1dfb-4cc6-a143-ea9827bc3dbd None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:29.561142 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea013f14-1dfb-4cc6-a143-ea9827bc3dbd None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:29.561142 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea013f14-1dfb-4cc6-a143-ea9827bc3dbd None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:29.562493 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 573/1135] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:07:29.568480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca30bec-3f49-4bf7-a19e-590801e65343 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:29.569014 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca30bec-3f49-4bf7-a19e-590801e65343 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:29.569014 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca30bec-3f49-4bf7-a19e-590801e65343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:29.588720 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-2ca30bec-3f49-4bf7-a19e-590801e65343 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:07:29.589951 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-2ca30bec-3f49-4bf7-a19e-590801e65343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:07:29.661697 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 563/1136] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:07:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:07:29.933151 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea013f14-1dfb-4cc6-a143-ea9827bc3dbd None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:30.020401 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:30.036411 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea013f14-1dfb-4cc6-a143-ea9827bc3dbd None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:30.070017 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595516e-3cee-4102-a614-996dd7cd4f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:30.070642 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595516e-3cee-4102-a614-996dd7cd4f73 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:30.070921 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595516e-3cee-4102-a614-996dd7cd4f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:30.071193 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595516e-3cee-4102-a614-996dd7cd4f73 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:30.072348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595516e-3cee-4102-a614-996dd7cd4f73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:30.073775 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595516e-3cee-4102-a614-996dd7cd4f73 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:30.076008 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 574/1137] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:07:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:30.081805 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca30bec-3f49-4bf7-a19e-590801e65343 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:30.114953 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:30.179426 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca30bec-3f49-4bf7-a19e-590801e65343 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:30.212695 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae0063-b480-4556-a2d4-02ad180175ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:30.213226 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae0063-b480-4556-a2d4-02ad180175ee None admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:30.213506 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae0063-b480-4556-a2d4-02ad180175ee None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:30.213781 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae0063-b480-4556-a2d4-02ad180175ee None admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:30.214429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae0063-b480-4556-a2d4-02ad180175ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:30.215995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae0063-b480-4556-a2d4-02ad180175ee None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:30.237217 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 564/1138] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Sep 5 12:07:30 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:07:32.174169 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52bbafa-a354-47ed-92fb-1ac5163df2b8 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:32.174704 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52bbafa-a354-47ed-92fb-1ac5163df2b8 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:32.174704 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52bbafa-a354-47ed-92fb-1ac5163df2b8 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:32.176097 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 575/1139] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:07:32.181388 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc59dfd-75c5-4186-a131-9bafa8043b03 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:32.181818 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc59dfd-75c5-4186-a131-9bafa8043b03 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:32.181912 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc59dfd-75c5-4186-a131-9bafa8043b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:32.200769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-1fc59dfd-75c5-4186-a131-9bafa8043b03 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:07:32.201939 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-1fc59dfd-75c5-4186-a131-9bafa8043b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:07:32.267785 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 565/1140] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:07:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:07:32.536707 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52bbafa-a354-47ed-92fb-1ac5163df2b8 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:32.635771 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52bbafa-a354-47ed-92fb-1ac5163df2b8 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:32.669140 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab54988-e5ff-4da3-88f0-3de855327f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:32.669760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab54988-e5ff-4da3-88f0-3de855327f06 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:32.670038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab54988-e5ff-4da3-88f0-3de855327f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:32.670308 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab54988-e5ff-4da3-88f0-3de855327f06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:32.680460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab54988-e5ff-4da3-88f0-3de855327f06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:07:32.685688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-0ab54988-e5ff-4da3-88f0-3de855327f06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:07:32.685688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:07:32.685688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:07:32.685688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:07:32.685688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:07:32.685688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:07:32.685688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:07:32.685688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:07:32.685688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:07:32.685688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:07:32.685688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:07:32.685688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:07:32.685688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:07:32.685688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:07:32.685688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:07:32.685688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:07:32.686801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:07:32.686801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:07:32.686801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:07:32.686801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:07:32.686801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:07:32.686801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:07:32.686801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:07:32.686801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:07:32.686801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:07:32.686801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:07:32.686801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:07:32.686801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:07:32.686801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:07:32.686801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:07:32.686801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:07:32.686801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:07:32.687926 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:07:32.687926 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:07:32.687926 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:07:32.687926 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:07:32.687926 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:07:32.687926 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:07:32.687926 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:07:32.687926 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:07:32.687926 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:07:32.687926 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:07:32.687926 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:07:32.687926 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:07:32.687926 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 576/1141] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:07:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:07:32.692799 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc59dfd-75c5-4186-a131-9bafa8043b03 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:32.716354 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:32.791221 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc59dfd-75c5-4186-a131-9bafa8043b03 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:32.824869 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f574c-b1d7-4505-bf29-d4fa43f3371e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:32.825468 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f574c-b1d7-4505-bf29-d4fa43f3371e None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:32.825749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f574c-b1d7-4505-bf29-d4fa43f3371e None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:32.826020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f574c-b1d7-4505-bf29-d4fa43f3371e None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:32.826691 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83f574c-b1d7-4505-bf29-d4fa43f3371e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:32.826996 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83f574c-b1d7-4505-bf29-d4fa43f3371e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:32.828378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83f574c-b1d7-4505-bf29-d4fa43f3371e None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:32.836231 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 566/1142] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:07:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:32.842711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab54988-e5ff-4da3-88f0-3de855327f06 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:32.861550 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:32.942167 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab54988-e5ff-4da3-88f0-3de855327f06 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:32.975147 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29947644-a60f-4abf-9fcf-8cb0a9f7d8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:32.975790 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29947644-a60f-4abf-9fcf-8cb0a9f7d8b9 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:32.976072 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29947644-a60f-4abf-9fcf-8cb0a9f7d8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:32.976343 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29947644-a60f-4abf-9fcf-8cb0a9f7d8b9 None admin] PATH_INFO: `/v3/users/65bd30779b604740a6af4753d08404ad` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:32.989706 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29947644-a60f-4abf-9fcf-8cb0a9f7d8b9 None admin] RBAC: Authorizing `identity:get_user(user_id=65bd30779b604740a6af4753d08404ad)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:32.991186 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29947644-a60f-4abf-9fcf-8cb0a9f7d8b9 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:32.992964 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 577/1143] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:07:32 2022] GET /identity/v3/users/65bd30779b604740a6af4753d08404ad => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:32.999157 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f574c-b1d7-4505-bf29-d4fa43f3371e None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:33.082347 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:33.098277 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f574c-b1d7-4505-bf29-d4fa43f3371e None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:33.130759 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f82129-daa2-4411-8ead-2f9ff3199729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:33.131350 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f82129-daa2-4411-8ead-2f9ff3199729 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:33.131718 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f82129-daa2-4411-8ead-2f9ff3199729 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:33.131987 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f82129-daa2-4411-8ead-2f9ff3199729 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:33.196793 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f82129-daa2-4411-8ead-2f9ff3199729 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:33.198218 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f82129-daa2-4411-8ead-2f9ff3199729 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:33.277277 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:33.294301 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 567/1144] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:07:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:07:33.300264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29947644-a60f-4abf-9fcf-8cb0a9f7d8b9 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:33.373461 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:33.398646 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29947644-a60f-4abf-9fcf-8cb0a9f7d8b9 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:33.431577 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2b580-90e2-4c10-8d85-e14063671d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:33.432195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2b580-90e2-4c10-8d85-e14063671d8f None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:33.432469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2b580-90e2-4c10-8d85-e14063671d8f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:33.432739 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2b580-90e2-4c10-8d85-e14063671d8f None admin] PATH_INFO: `/v3/projects/fdc41e3f67fb447584c24be83d6d8733` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:33.433858 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f2b580-90e2-4c10-8d85-e14063671d8f None admin] RBAC: Authorizing `identity:get_project(project_id=fdc41e3f67fb447584c24be83d6d8733)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:33.435316 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f2b580-90e2-4c10-8d85-e14063671d8f None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:33.436972 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 578/1145] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:07:33 2022] GET /identity/v3/projects/fdc41e3f67fb447584c24be83d6d8733 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:33.443135 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f82129-daa2-4411-8ead-2f9ff3199729 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:33.543577 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f82129-daa2-4411-8ead-2f9ff3199729 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:33.577647 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc811b13-104b-4d33-b0e7-4def6a2e8dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:33.578239 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc811b13-104b-4d33-b0e7-4def6a2e8dce None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:33.578518 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc811b13-104b-4d33-b0e7-4def6a2e8dce None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:33.578793 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc811b13-104b-4d33-b0e7-4def6a2e8dce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:33.579512 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc811b13-104b-4d33-b0e7-4def6a2e8dce None admin] RBAC: Adding query filter params (role.id=618c48706f7f445abeddf7eb2f8e927a, scope.project.id=fdc41e3f67fb447584c24be83d6d8733, user.id=65bd30779b604740a6af4753d08404ad) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:33.579851 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc811b13-104b-4d33-b0e7-4def6a2e8dce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:33.581261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc811b13-104b-4d33-b0e7-4def6a2e8dce None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:33.586662 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 568/1146] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:07:33 2022] GET /identity/v3/role_assignments?user.id=65bd30779b604740a6af4753d08404ad&scope.project.id=fdc41e3f67fb447584c24be83d6d8733&role.id=618c48706f7f445abeddf7eb2f8e927a => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:35.453786 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b41ef2-cee5-486d-9014-f1806ed969c3 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:35.454309 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b41ef2-cee5-486d-9014-f1806ed969c3 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:35.454309 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b41ef2-cee5-486d-9014-f1806ed969c3 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:35.455678 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 579/1147] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:07:35.460756 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f347eb-346f-44f4-b306-95574d82b0e4 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:35.461151 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f347eb-346f-44f4-b306-95574d82b0e4 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:35.461299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f347eb-346f-44f4-b306-95574d82b0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:35.479402 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-55f347eb-346f-44f4-b306-95574d82b0e4 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:07:35.481113 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-55f347eb-346f-44f4-b306-95574d82b0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:07:35.524574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:35.547995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 569/1148] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:07:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:07:35.816529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b41ef2-cee5-486d-9014-f1806ed969c3 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:35.917371 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b41ef2-cee5-486d-9014-f1806ed969c3 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:35.949715 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d62251d-5c4d-48d5-922b-ed89acf25500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:35.950328 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d62251d-5c4d-48d5-922b-ed89acf25500 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:35.950607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d62251d-5c4d-48d5-922b-ed89acf25500 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:35.950880 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d62251d-5c4d-48d5-922b-ed89acf25500 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:35.960483 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d62251d-5c4d-48d5-922b-ed89acf25500 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:07:35.965659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-9d62251d-5c4d-48d5-922b-ed89acf25500 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:07:35.965659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:07:35.965659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:07:35.965659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:07:35.965659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:07:35.965659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:07:35.965659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:07:35.965659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:07:35.965659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:07:35.965659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:07:35.965659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:07:35.965659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:07:35.965659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:07:35.965659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:07:35.965659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:07:35.965659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:07:35.966782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:07:35.966782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:07:35.966782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:07:35.966782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:07:35.966782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:07:35.966782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:07:35.966782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:07:35.966782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:07:35.966782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:07:35.966782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:07:35.966782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:07:35.966782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:07:35.966782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:07:35.966782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:07:35.966782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:07:35.966782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:07:35.967884 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:07:35.967884 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:07:35.967884 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:07:35.967884 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:07:35.967884 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:07:35.967884 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:07:35.967884 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:07:35.967884 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:07:35.967884 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:07:35.967884 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:07:35.967884 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:07:35.967884 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:07:35.967884 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 580/1149] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:07:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:07:35.972952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f347eb-346f-44f4-b306-95574d82b0e4 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:36.070731 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f347eb-346f-44f4-b306-95574d82b0e4 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:36.103784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c21d4-e424-4427-a1ed-2266b9699612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:36.104386 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c21d4-e424-4427-a1ed-2266b9699612 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:36.104664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c21d4-e424-4427-a1ed-2266b9699612 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:36.104936 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c21d4-e424-4427-a1ed-2266b9699612 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:36.106213 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495c21d4-e424-4427-a1ed-2266b9699612 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:36.106468 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495c21d4-e424-4427-a1ed-2266b9699612 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:36.107855 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495c21d4-e424-4427-a1ed-2266b9699612 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:36.116518 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 570/1150] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:07:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:36.124348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d62251d-5c4d-48d5-922b-ed89acf25500 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:36.127775 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:36.222606 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d62251d-5c4d-48d5-922b-ed89acf25500 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:36.255371 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38775049-fa12-4cfb-9a4e-aacc4b7a2e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:36.256022 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38775049-fa12-4cfb-9a4e-aacc4b7a2e49 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:36.256300 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38775049-fa12-4cfb-9a4e-aacc4b7a2e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:36.256574 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38775049-fa12-4cfb-9a4e-aacc4b7a2e49 None admin] PATH_INFO: `/v3/users/65bd30779b604740a6af4753d08404ad` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:36.257671 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38775049-fa12-4cfb-9a4e-aacc4b7a2e49 None admin] RBAC: Authorizing `identity:get_user(user_id=65bd30779b604740a6af4753d08404ad)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:36.259091 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38775049-fa12-4cfb-9a4e-aacc4b7a2e49 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:36.260696 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 581/1151] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:07:36 2022] GET /identity/v3/users/65bd30779b604740a6af4753d08404ad => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:36.267054 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c21d4-e424-4427-a1ed-2266b9699612 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:36.308266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:36.365734 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c21d4-e424-4427-a1ed-2266b9699612 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:36.398705 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ef35c-939c-40b6-801f-241c209412f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:36.399327 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ef35c-939c-40b6-801f-241c209412f9 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:36.399635 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ef35c-939c-40b6-801f-241c209412f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:36.399937 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ef35c-939c-40b6-801f-241c209412f9 None admin] PATH_INFO: `/v3/projects/fdc41e3f67fb447584c24be83d6d8733` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:36.401043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6ef35c-939c-40b6-801f-241c209412f9 None admin] RBAC: Authorizing `identity:get_project(project_id=fdc41e3f67fb447584c24be83d6d8733)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:36.402518 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6ef35c-939c-40b6-801f-241c209412f9 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:36.404892 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 571/1152] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:07:36 2022] GET /identity/v3/projects/fdc41e3f67fb447584c24be83d6d8733 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:36.410872 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38775049-fa12-4cfb-9a4e-aacc4b7a2e49 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:36.478450 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:36.509101 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38775049-fa12-4cfb-9a4e-aacc4b7a2e49 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:36.542541 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255cc0d4-5be8-447d-ad23-f45159bfc2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:36.543230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255cc0d4-5be8-447d-ad23-f45159bfc2c2 None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:36.543511 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255cc0d4-5be8-447d-ad23-f45159bfc2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:36.543816 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255cc0d4-5be8-447d-ad23-f45159bfc2c2 None admin] PATH_INFO: `/v3/projects/fdc41e3f67fb447584c24be83d6d8733/users/65bd30779b604740a6af4753d08404ad/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:36.545691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255cc0d4-5be8-447d-ad23-f45159bfc2c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=fdc41e3f67fb447584c24be83d6d8733, user_id=65bd30779b604740a6af4753d08404ad, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:36.547119 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255cc0d4-5be8-447d-ad23-f45159bfc2c2 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:36.560444 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 582/1153] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Sep 5 12:07:36 2022] PUT /identity/v3/projects/fdc41e3f67fb447584c24be83d6d8733/users/65bd30779b604740a6af4753d08404ad/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:07:38.486136 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee694365-dc7d-4558-8c13-311dbb6e59d2 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:38.486918 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee694365-dc7d-4558-8c13-311dbb6e59d2 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:38.486918 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee694365-dc7d-4558-8c13-311dbb6e59d2 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:38.488058 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 572/1154] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:07:38.493382 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a10030e-823c-44c1-88fa-2c21ee8bee17 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:38.493883 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a10030e-823c-44c1-88fa-2c21ee8bee17 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:38.493986 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a10030e-823c-44c1-88fa-2c21ee8bee17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:38.512745 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-8a10030e-823c-44c1-88fa-2c21ee8bee17 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:07:38.513979 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8a10030e-823c-44c1-88fa-2c21ee8bee17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:07:38.583383 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 583/1155] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:07:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:07:38.851953 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee694365-dc7d-4558-8c13-311dbb6e59d2 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:38.956600 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee694365-dc7d-4558-8c13-311dbb6e59d2 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:38.965902 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:38.990378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb3c3c2-0311-4b85-b9db-786b4867d8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:38.990996 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3c3c2-0311-4b85-b9db-786b4867d8cd None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:38.991268 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3c3c2-0311-4b85-b9db-786b4867d8cd None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:38.991554 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3c3c2-0311-4b85-b9db-786b4867d8cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:39.002439 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb3c3c2-0311-4b85-b9db-786b4867d8cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:07:39.007181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-0fb3c3c2-0311-4b85-b9db-786b4867d8cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:07:39.007181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:07:39.007181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:07:39.007181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:07:39.007181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:07:39.007181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:07:39.007181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:07:39.007181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:07:39.007181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:07:39.007181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:07:39.007181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:07:39.007181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:07:39.007181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:07:39.007181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:07:39.007181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:07:39.007181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:07:39.009046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:07:39.009046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:07:39.009046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:07:39.009046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:07:39.009046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:07:39.009046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:07:39.009046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:07:39.009046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:07:39.009046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:07:39.009046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:07:39.009046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:07:39.009046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:07:39.009046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:07:39.009046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:07:39.009046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:07:39.009046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:07:39.010851 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:07:39.010851 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:07:39.010851 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:07:39.010851 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:07:39.010851 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:07:39.010851 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:07:39.010851 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:07:39.010851 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:07:39.010851 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:07:39.010851 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:07:39.010851 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:07:39.010851 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:07:39.010851 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 573/1156] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:07:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:07:39.015118 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a10030e-823c-44c1-88fa-2c21ee8bee17 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:39.076387 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:39.116932 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a10030e-823c-44c1-88fa-2c21ee8bee17 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:39.151271 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebef32-bf05-45b3-979d-ed635fc8e6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:39.151923 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebef32-bf05-45b3-979d-ed635fc8e6bc None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:39.152198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebef32-bf05-45b3-979d-ed635fc8e6bc None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:39.152505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebef32-bf05-45b3-979d-ed635fc8e6bc None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:39.153420 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ebef32-bf05-45b3-979d-ed635fc8e6bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:39.153720 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ebef32-bf05-45b3-979d-ed635fc8e6bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:39.155096 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ebef32-bf05-45b3-979d-ed635fc8e6bc None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:39.163422 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 584/1157] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:07:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:39.169860 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb3c3c2-0311-4b85-b9db-786b4867d8cd None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:39.269557 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb3c3c2-0311-4b85-b9db-786b4867d8cd None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:39.302732 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ce942-ef2c-4edc-88e8-580f37f37814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:39.303345 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ce942-ef2c-4edc-88e8-580f37f37814 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:39.303652 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ce942-ef2c-4edc-88e8-580f37f37814 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:39.303928 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ce942-ef2c-4edc-88e8-580f37f37814 None admin] PATH_INFO: `/v3/users/65bd30779b604740a6af4753d08404ad` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:39.305029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ce942-ef2c-4edc-88e8-580f37f37814 None admin] RBAC: Authorizing `identity:get_user(user_id=65bd30779b604740a6af4753d08404ad)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:39.306446 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ce942-ef2c-4edc-88e8-580f37f37814 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:39.308126 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 574/1158] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:07:39 2022] GET /identity/v3/users/65bd30779b604740a6af4753d08404ad => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:39.314929 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebef32-bf05-45b3-979d-ed635fc8e6bc None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:39.414738 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebef32-bf05-45b3-979d-ed635fc8e6bc None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:39.440542 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:39.452485 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78c4db-5111-4682-b3b0-0d596a400d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:39.453092 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78c4db-5111-4682-b3b0-0d596a400d08 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:39.453369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78c4db-5111-4682-b3b0-0d596a400d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:39.453638 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78c4db-5111-4682-b3b0-0d596a400d08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:39.522679 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78c4db-5111-4682-b3b0-0d596a400d08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:39.524183 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78c4db-5111-4682-b3b0-0d596a400d08 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:39.630551 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 585/1159] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:07:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 05 12:07:39.636790 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ce942-ef2c-4edc-88e8-580f37f37814 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:39.672606 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:39.737835 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ce942-ef2c-4edc-88e8-580f37f37814 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:39.771137 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355ca00-2e7c-4ebf-80db-d939fce3e9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:39.771783 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355ca00-2e7c-4ebf-80db-d939fce3e9e1 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:39.772055 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355ca00-2e7c-4ebf-80db-d939fce3e9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:39.772319 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355ca00-2e7c-4ebf-80db-d939fce3e9e1 None admin] PATH_INFO: `/v3/projects/fdc41e3f67fb447584c24be83d6d8733` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:39.773422 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e355ca00-2e7c-4ebf-80db-d939fce3e9e1 None admin] RBAC: Authorizing `identity:get_project(project_id=fdc41e3f67fb447584c24be83d6d8733)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:39.774849 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e355ca00-2e7c-4ebf-80db-d939fce3e9e1 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:39.776506 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 575/1160] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:07:39 2022] GET /identity/v3/projects/fdc41e3f67fb447584c24be83d6d8733 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:39.783812 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78c4db-5111-4682-b3b0-0d596a400d08 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:39.807915 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:39.882295 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78c4db-5111-4682-b3b0-0d596a400d08 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:39.918835 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9336ec-7c16-46f0-b33c-c1e7af6e9aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:39.919439 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9336ec-7c16-46f0-b33c-c1e7af6e9aed None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:39.919749 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9336ec-7c16-46f0-b33c-c1e7af6e9aed None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:39.920023 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9336ec-7c16-46f0-b33c-c1e7af6e9aed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:39.920756 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9336ec-7c16-46f0-b33c-c1e7af6e9aed None admin] RBAC: Adding query filter params (role.id=618c48706f7f445abeddf7eb2f8e927a, scope.project.id=fdc41e3f67fb447584c24be83d6d8733, user.id=65bd30779b604740a6af4753d08404ad) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:39.921427 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9336ec-7c16-46f0-b33c-c1e7af6e9aed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:39.922859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9336ec-7c16-46f0-b33c-c1e7af6e9aed None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:39.928932 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 586/1161] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:07:39 2022] GET /identity/v3/role_assignments?user.id=65bd30779b604740a6af4753d08404ad&scope.project.id=fdc41e3f67fb447584c24be83d6d8733&role.id=618c48706f7f445abeddf7eb2f8e927a => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:41.873100 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ab1a6-59e6-49ee-b615-b74893b3d861 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:41.873692 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ab1a6-59e6-49ee-b615-b74893b3d861 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:41.873692 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ab1a6-59e6-49ee-b615-b74893b3d861 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:41.874973 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 576/1162] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:07:41.880564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f810c6-15f0-4def-bf0c-fe054231fda3 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:41.881025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f810c6-15f0-4def-bf0c-fe054231fda3 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:41.881125 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f810c6-15f0-4def-bf0c-fe054231fda3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:41.899816 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-42f810c6-15f0-4def-bf0c-fe054231fda3 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:07:41.901031 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-42f810c6-15f0-4def-bf0c-fe054231fda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:07:41.971417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 587/1163] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:07:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:07:42.250793 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ab1a6-59e6-49ee-b615-b74893b3d861 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:42.361567 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ab1a6-59e6-49ee-b615-b74893b3d861 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:42.364060 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:42.396432 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee666f-50b5-4747-8239-52d1aa5b40d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:42.397040 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee666f-50b5-4747-8239-52d1aa5b40d6 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:42.397316 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee666f-50b5-4747-8239-52d1aa5b40d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:42.397586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee666f-50b5-4747-8239-52d1aa5b40d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:42.463896 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee666f-50b5-4747-8239-52d1aa5b40d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:42.465419 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee666f-50b5-4747-8239-52d1aa5b40d6 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:42.567556 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 577/1164] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:07:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:07:42.573684 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f810c6-15f0-4def-bf0c-fe054231fda3 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:42.591201 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:42.682852 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f810c6-15f0-4def-bf0c-fe054231fda3 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:42.732633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0281b5-5716-4179-8052-4a0c0de471df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:42.733267 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0281b5-5716-4179-8052-4a0c0de471df None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:42.733548 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0281b5-5716-4179-8052-4a0c0de471df None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:42.733822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0281b5-5716-4179-8052-4a0c0de471df None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:42.741058 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0281b5-5716-4179-8052-4a0c0de471df None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:42.742526 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0281b5-5716-4179-8052-4a0c0de471df None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:42.755555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-0c0281b5-5716-4179-8052-4a0c0de471df None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 05 12:07:42.755555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:07:42.755555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 05 12:07:42.755555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 05 12:07:42.755555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:07:42.755555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:07:42.755555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:07:42.755555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:07:42.755555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 05 12:07:42.755555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:07:42.755555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:07:42.755555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:07:42.755555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:07:42.755555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:07:42.755555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:07:42.755555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:07:42.755555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:07:42.756886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:07:42.756886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:07:42.756886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:07:42.756886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:07:42.756886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:07:42.756886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:07:42.756886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:07:42.756886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:07:42.756886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:07:42.756886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:07:42.756886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:07:42.756886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:07:42.756886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:07:42.756886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:07:42.756886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:07:42.756886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:07:42.757973 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:07:42.757973 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:07:42.757973 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:07:42.757973 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:07:42.757973 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:07:42.757973 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:07:42.757973 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:07:42.757973 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:07:42.757973 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:07:42.757973 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:07:42.757973 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 05 12:07:42.757973 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:07:42.757973 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 05 12:07:42.757973 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:07:42.757973 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 588/1165] 10.222.0.16 () {56 vars in 1117 bytes} [Mon Sep 5 12:07:42 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:07:42.766893 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee666f-50b5-4747-8239-52d1aa5b40d6 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:42.770180 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:42.878816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee666f-50b5-4747-8239-52d1aa5b40d6 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:42.915832 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9dd519-7cb6-49da-a36b-09052b664ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:42.916487 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9dd519-7cb6-49da-a36b-09052b664ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:42.916788 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9dd519-7cb6-49da-a36b-09052b664ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:42.917062 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9dd519-7cb6-49da-a36b-09052b664ed8 None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:42.917763 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9dd519-7cb6-49da-a36b-09052b664ed8 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:42.918119 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9dd519-7cb6-49da-a36b-09052b664ed8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:42.919602 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9dd519-7cb6-49da-a36b-09052b664ed8 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:42.927504 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 578/1166] 10.222.0.16 () {56 vars in 1115 bytes} [Mon Sep 5 12:07:42 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:45.018829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d50bf3-b377-4750-aaa8-97fdb302ef46 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:45.019378 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d50bf3-b377-4750-aaa8-97fdb302ef46 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:45.019378 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d50bf3-b377-4750-aaa8-97fdb302ef46 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:45.020969 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 589/1167] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:07:45.028230 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3944344-c42b-4987-8a70-2aa4dd783c7b None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:45.028741 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3944344-c42b-4987-8a70-2aa4dd783c7b None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:45.028864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3944344-c42b-4987-8a70-2aa4dd783c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:45.050960 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-e3944344-c42b-4987-8a70-2aa4dd783c7b None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:07:45.052190 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-e3944344-c42b-4987-8a70-2aa4dd783c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:07:45.139786 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:45.155029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 579/1168] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:07:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:07:45.458494 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d50bf3-b377-4750-aaa8-97fdb302ef46 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:45.542978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:45.576416 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d50bf3-b377-4750-aaa8-97fdb302ef46 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:45.616826 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ceb594-192d-4fcb-ac31-fd13a76dbbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:45.617486 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ceb594-192d-4fcb-ac31-fd13a76dbbc7 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:45.617766 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ceb594-192d-4fcb-ac31-fd13a76dbbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:45.618078 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ceb594-192d-4fcb-ac31-fd13a76dbbc7 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:45.618786 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ceb594-192d-4fcb-ac31-fd13a76dbbc7 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:45.620385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ceb594-192d-4fcb-ac31-fd13a76dbbc7 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:45.639996 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 590/1169] 10.222.0.16 () {60 vars in 1138 bytes} [Mon Sep 5 12:07:45 2022] POST /identity/v3/domains => generated 267 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:07:47.661011 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee843e-0b41-480e-83ac-f494f7621a7c None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:47.661529 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee843e-0b41-480e-83ac-f494f7621a7c None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:47.661529 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee843e-0b41-480e-83ac-f494f7621a7c None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:47.663055 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 580/1170] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:07:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:07:47.669682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb0f1b-b625-483c-a84a-bfca6ff69ffc None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:47.670152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb0f1b-b625-483c-a84a-bfca6ff69ffc None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:47.670242 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb0f1b-b625-483c-a84a-bfca6ff69ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:47.689936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-bdcb0f1b-b625-483c-a84a-bfca6ff69ffc None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:07:47.691181 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-bdcb0f1b-b625-483c-a84a-bfca6ff69ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:07:47.766664 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 591/1171] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:07:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:07:48.065145 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee843e-0b41-480e-83ac-f494f7621a7c None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:48.180514 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee843e-0b41-480e-83ac-f494f7621a7c None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:48.219611 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce816cf-13dc-4384-ba83-a587068b4ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:48.220245 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce816cf-13dc-4384-ba83-a587068b4ada None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:48.220524 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce816cf-13dc-4384-ba83-a587068b4ada None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:48.220796 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce816cf-13dc-4384-ba83-a587068b4ada None admin] PATH_INFO: `/v3/domains/616150a904874a82a7570cc57d84e743` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:48.232785 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce816cf-13dc-4384-ba83-a587068b4ada None admin] RBAC: Authorizing `identity:get_domain(domain_id=616150a904874a82a7570cc57d84e743)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:48.234499 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce816cf-13dc-4384-ba83-a587068b4ada None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:48.236441 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 581/1172] 10.222.0.16 () {56 vars in 1183 bytes} [Mon Sep 5 12:07:48 2022] GET /identity/v3/domains/616150a904874a82a7570cc57d84e743 => generated 267 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:48.243880 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb0f1b-b625-483c-a84a-bfca6ff69ffc None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:48.314500 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:48.352905 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb0f1b-b625-483c-a84a-bfca6ff69ffc None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:48.390079 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33c381-26f4-4302-8611-d66e404bd5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:48.390732 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33c381-26f4-4302-8611-d66e404bd5e1 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:48.391004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33c381-26f4-4302-8611-d66e404bd5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:48.391348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33c381-26f4-4302-8611-d66e404bd5e1 None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:48.392083 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df33c381-26f4-4302-8611-d66e404bd5e1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:48.393523 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df33c381-26f4-4302-8611-d66e404bd5e1 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:48.426058 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 592/1173] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Sep 5 12:07:48 2022] POST /identity/v3/projects => generated 361 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:07:50.634555 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012ba24-9d7c-4acc-8b4d-246b43165702 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:50.635196 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012ba24-9d7c-4acc-8b4d-246b43165702 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:50.635196 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012ba24-9d7c-4acc-8b4d-246b43165702 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:50.636352 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 582/1174] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:07:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:07:50.642572 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754af21-308c-4a4d-9fd9-40f53e8ee628 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:50.643047 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754af21-308c-4a4d-9fd9-40f53e8ee628 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:50.643195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754af21-308c-4a4d-9fd9-40f53e8ee628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:50.662902 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-5754af21-308c-4a4d-9fd9-40f53e8ee628 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:07:50.664317 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5754af21-308c-4a4d-9fd9-40f53e8ee628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:07:50.737705 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 593/1175] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:07:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:07:51.044908 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012ba24-9d7c-4acc-8b4d-246b43165702 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:51.116398 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:51.179788 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012ba24-9d7c-4acc-8b4d-246b43165702 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:51.229606 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae904384-8948-4173-aca0-de2b438b8e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:51.230272 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae904384-8948-4173-aca0-de2b438b8e20 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:51.230740 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae904384-8948-4173-aca0-de2b438b8e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:51.231023 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae904384-8948-4173-aca0-de2b438b8e20 None admin] PATH_INFO: `/v3/domains/616150a904874a82a7570cc57d84e743` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:51.232340 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae904384-8948-4173-aca0-de2b438b8e20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=616150a904874a82a7570cc57d84e743)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:51.233829 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae904384-8948-4173-aca0-de2b438b8e20 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:51.235536 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 583/1176] 10.222.0.16 () {56 vars in 1183 bytes} [Mon Sep 5 12:07:51 2022] GET /identity/v3/domains/616150a904874a82a7570cc57d84e743 => generated 267 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:51.241634 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754af21-308c-4a4d-9fd9-40f53e8ee628 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:51.299625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:51.360156 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754af21-308c-4a4d-9fd9-40f53e8ee628 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:51.397786 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0d3fc-5d10-4160-8d80-e50a7650fcf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:51.398409 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0d3fc-5d10-4160-8d80-e50a7650fcf5 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:51.398689 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0d3fc-5d10-4160-8d80-e50a7650fcf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:51.398963 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0d3fc-5d10-4160-8d80-e50a7650fcf5 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:51.399692 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e0d3fc-5d10-4160-8d80-e50a7650fcf5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:51.401156 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e0d3fc-5d10-4160-8d80-e50a7650fcf5 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:51.426707 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 594/1177] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Sep 5 12:07:51 2022] POST /identity/v3/users => generated 311 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:07:53.391273 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b324b-33ef-4447-b943-501d0cdfabf2 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:53.391798 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b324b-33ef-4447-b943-501d0cdfabf2 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:53.391892 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b324b-33ef-4447-b943-501d0cdfabf2 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:53.393172 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 584/1178] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:07:53.398412 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede6d57-0691-401d-bdbe-3c25bf31d1d5 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:53.398854 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede6d57-0691-401d-bdbe-3c25bf31d1d5 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:53.398947 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede6d57-0691-401d-bdbe-3c25bf31d1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:53.417679 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-aede6d57-0691-401d-bdbe-3c25bf31d1d5 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:07:53.418928 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-aede6d57-0691-401d-bdbe-3c25bf31d1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:07:53.487776 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 595/1179] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:07:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:07:53.753703 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b324b-33ef-4447-b943-501d0cdfabf2 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:53.855967 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b324b-33ef-4447-b943-501d0cdfabf2 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:53.881465 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:53.890999 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fba3904-d8ea-4f65-b597-91c94d1c2020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:53.891641 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba3904-d8ea-4f65-b597-91c94d1c2020 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:53.891916 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba3904-d8ea-4f65-b597-91c94d1c2020 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:53.892181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba3904-d8ea-4f65-b597-91c94d1c2020 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:53.901933 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fba3904-d8ea-4f65-b597-91c94d1c2020 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:07:53.906961 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-2fba3904-d8ea-4f65-b597-91c94d1c2020 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:07:53.906961 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:07:53.906961 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:07:53.906961 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:07:53.906961 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:07:53.906961 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:07:53.906961 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:07:53.906961 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:07:53.906961 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:07:53.906961 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:07:53.906961 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:07:53.906961 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:07:53.906961 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:07:53.906961 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:07:53.906961 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:07:53.906961 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:07:53.908043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:07:53.908043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:07:53.908043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:07:53.908043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:07:53.908043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:07:53.908043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:07:53.908043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:07:53.908043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:07:53.908043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:07:53.908043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:07:53.908043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:07:53.908043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:07:53.908043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:07:53.908043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:07:53.908043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:07:53.908043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:07:53.909305 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:07:53.909305 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:07:53.909305 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:07:53.909305 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:07:53.909305 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:07:53.909305 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:07:53.909305 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:07:53.909305 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:07:53.909305 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:07:53.909305 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:07:53.909305 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:07:53.909305 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:07:53.909305 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 585/1180] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:07:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:07:53.914121 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede6d57-0691-401d-bdbe-3c25bf31d1d5 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:53.947439 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:54.012723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede6d57-0691-401d-bdbe-3c25bf31d1d5 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:54.045300 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5918b-e4a7-45e7-b275-005030a4ca35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:54.045903 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5918b-e4a7-45e7-b275-005030a4ca35 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:54.046185 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5918b-e4a7-45e7-b275-005030a4ca35 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:54.046458 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5918b-e4a7-45e7-b275-005030a4ca35 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:54.047115 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5918b-e4a7-45e7-b275-005030a4ca35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:54.047418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5918b-e4a7-45e7-b275-005030a4ca35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:54.048823 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5918b-e4a7-45e7-b275-005030a4ca35 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:54.057126 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 596/1181] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:07:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:54.064102 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fba3904-d8ea-4f65-b597-91c94d1c2020 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:54.173477 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fba3904-d8ea-4f65-b597-91c94d1c2020 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:54.212399 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8f5b5-fb6a-4138-b5ab-9616e5f8960f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:54.213044 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8f5b5-fb6a-4138-b5ab-9616e5f8960f None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:54.213348 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8f5b5-fb6a-4138-b5ab-9616e5f8960f None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:54.213633 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8f5b5-fb6a-4138-b5ab-9616e5f8960f None admin] PATH_INFO: `/v3/users/2590744c87dc4895ad1c14a48166f969` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:54.228640 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa8f5b5-fb6a-4138-b5ab-9616e5f8960f None admin] RBAC: Authorizing `identity:get_user(user_id=2590744c87dc4895ad1c14a48166f969)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:54.230076 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa8f5b5-fb6a-4138-b5ab-9616e5f8960f None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:54.232380 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 586/1182] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:07:54 2022] GET /identity/v3/users/2590744c87dc4895ad1c14a48166f969 => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:54.238587 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5918b-e4a7-45e7-b275-005030a4ca35 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:54.330811 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:54.337148 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5918b-e4a7-45e7-b275-005030a4ca35 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:54.370062 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b7b1b-5245-44b1-9e76-49a6d97e1d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:54.370670 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b7b1b-5245-44b1-9e76-49a6d97e1d66 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:54.370947 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b7b1b-5245-44b1-9e76-49a6d97e1d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:54.371883 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b7b1b-5245-44b1-9e76-49a6d97e1d66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:54.437385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9b7b1b-5245-44b1-9e76-49a6d97e1d66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:54.438830 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9b7b1b-5245-44b1-9e76-49a6d97e1d66 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:54.528300 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:54.535162 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 597/1183] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:07:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:07:54.541312 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8f5b5-fb6a-4138-b5ab-9616e5f8960f None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:54.586619 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:54.640207 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8f5b5-fb6a-4138-b5ab-9616e5f8960f None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:54.674460 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aeffdc-0222-4630-93c9-96cd60f36bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:54.675086 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aeffdc-0222-4630-93c9-96cd60f36bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:54.675365 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aeffdc-0222-4630-93c9-96cd60f36bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:54.675676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aeffdc-0222-4630-93c9-96cd60f36bf2 None admin] PATH_INFO: `/v3/projects/96270b49feb741de94c788f1d9e65eb3` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:54.676795 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aeffdc-0222-4630-93c9-96cd60f36bf2 None admin] RBAC: Authorizing `identity:get_project(project_id=96270b49feb741de94c788f1d9e65eb3)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:54.678217 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aeffdc-0222-4630-93c9-96cd60f36bf2 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:54.679877 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 587/1184] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:07:54 2022] GET /identity/v3/projects/96270b49feb741de94c788f1d9e65eb3 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:54.686184 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b7b1b-5245-44b1-9e76-49a6d97e1d66 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:54.783702 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b7b1b-5245-44b1-9e76-49a6d97e1d66 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:54.820373 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba066ed8-e9e3-4c7b-88d4-3a91009307a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:54.820985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba066ed8-e9e3-4c7b-88d4-3a91009307a1 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:54.821264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba066ed8-e9e3-4c7b-88d4-3a91009307a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:54.821539 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba066ed8-e9e3-4c7b-88d4-3a91009307a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:54.822289 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba066ed8-e9e3-4c7b-88d4-3a91009307a1 None admin] RBAC: Adding query filter params (role.id=618c48706f7f445abeddf7eb2f8e927a, scope.project.id=96270b49feb741de94c788f1d9e65eb3, user.id=2590744c87dc4895ad1c14a48166f969) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:54.822589 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba066ed8-e9e3-4c7b-88d4-3a91009307a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:54.824110 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba066ed8-e9e3-4c7b-88d4-3a91009307a1 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:54.830391 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 598/1185] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:07:54 2022] GET /identity/v3/role_assignments?user.id=2590744c87dc4895ad1c14a48166f969&scope.project.id=96270b49feb741de94c788f1d9e65eb3&role.id=618c48706f7f445abeddf7eb2f8e927a => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:56.732077 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65ade2-2922-4550-8cae-ad0e80861cfe None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:56.732576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65ade2-2922-4550-8cae-ad0e80861cfe None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:56.732576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65ade2-2922-4550-8cae-ad0e80861cfe None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:56.733866 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 588/1186] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:07:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:07:56.739098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935a196-ac72-40e9-a417-438b0135fc74 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:56.739485 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935a196-ac72-40e9-a417-438b0135fc74 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:56.739671 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935a196-ac72-40e9-a417-438b0135fc74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:56.757516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-4935a196-ac72-40e9-a417-438b0135fc74 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:07:56.758681 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4935a196-ac72-40e9-a417-438b0135fc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:07:56.816137 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:56.827936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 599/1187] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:07:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:07:57.102688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65ade2-2922-4550-8cae-ad0e80861cfe None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:57.206314 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65ade2-2922-4550-8cae-ad0e80861cfe None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:57.219274 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:57.240989 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c221e-a24b-4a91-acf8-202dbd3f1c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:57.241607 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c221e-a24b-4a91-acf8-202dbd3f1c4a None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:57.241887 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c221e-a24b-4a91-acf8-202dbd3f1c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:57.242160 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c221e-a24b-4a91-acf8-202dbd3f1c4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:57.253072 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887c221e-a24b-4a91-acf8-202dbd3f1c4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:07:57.257904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-887c221e-a24b-4a91-acf8-202dbd3f1c4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:07:57.257904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:07:57.257904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:07:57.257904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:07:57.257904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:07:57.257904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:07:57.257904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:07:57.257904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:07:57.257904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:07:57.257904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:07:57.257904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:07:57.257904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:07:57.257904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:07:57.257904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:07:57.257904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:07:57.257904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:07:57.259058 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:07:57.259058 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:07:57.259058 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:07:57.259058 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:07:57.259058 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:07:57.259058 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:07:57.259058 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:07:57.259058 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:07:57.259058 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:07:57.259058 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:07:57.259058 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:07:57.259058 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:07:57.259058 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:07:57.259058 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:07:57.259058 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:07:57.259058 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:07:57.259992 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:07:57.259992 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:07:57.259992 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:07:57.259992 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:07:57.259992 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:07:57.259992 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:07:57.259992 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:07:57.259992 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:07:57.259992 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:07:57.259992 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:07:57.259992 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:07:57.259992 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:07:57.259992 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 589/1188] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:07:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:07:57.265090 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4935a196-ac72-40e9-a417-438b0135fc74 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:57.360679 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4935a196-ac72-40e9-a417-438b0135fc74 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:57.393724 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e67cac9-ef2f-43a4-9f9c-d7ddbfdf25c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:57.394365 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67cac9-ef2f-43a4-9f9c-d7ddbfdf25c6 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:57.394625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67cac9-ef2f-43a4-9f9c-d7ddbfdf25c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:57.394883 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67cac9-ef2f-43a4-9f9c-d7ddbfdf25c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:57.395548 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e67cac9-ef2f-43a4-9f9c-d7ddbfdf25c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:07:57.395863 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e67cac9-ef2f-43a4-9f9c-d7ddbfdf25c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:57.397213 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e67cac9-ef2f-43a4-9f9c-d7ddbfdf25c6 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:57.405297 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 600/1189] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:07:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:57.411133 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c221e-a24b-4a91-acf8-202dbd3f1c4a None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:57.508817 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c221e-a24b-4a91-acf8-202dbd3f1c4a None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:57.541895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328d8f7-f0e6-4634-bfd7-830fc9be54fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:57.542506 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328d8f7-f0e6-4634-bfd7-830fc9be54fc None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:57.542786 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328d8f7-f0e6-4634-bfd7-830fc9be54fc None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:57.543644 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328d8f7-f0e6-4634-bfd7-830fc9be54fc None admin] PATH_INFO: `/v3/users/2590744c87dc4895ad1c14a48166f969` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:57.544766 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2328d8f7-f0e6-4634-bfd7-830fc9be54fc None admin] RBAC: Authorizing `identity:get_user(user_id=2590744c87dc4895ad1c14a48166f969)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:57.546193 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2328d8f7-f0e6-4634-bfd7-830fc9be54fc None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:57.547785 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 590/1190] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:07:57 2022] GET /identity/v3/users/2590744c87dc4895ad1c14a48166f969 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:57.553742 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e67cac9-ef2f-43a4-9f9c-d7ddbfdf25c6 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:57.604565 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:57.651990 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e67cac9-ef2f-43a4-9f9c-d7ddbfdf25c6 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:57.689623 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef13c73-8951-4209-b0e0-c4df90d6a7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:57.690253 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef13c73-8951-4209-b0e0-c4df90d6a7ef None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:57.690533 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef13c73-8951-4209-b0e0-c4df90d6a7ef None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:57.690801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef13c73-8951-4209-b0e0-c4df90d6a7ef None admin] PATH_INFO: `/v3/projects/96270b49feb741de94c788f1d9e65eb3` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:57.691929 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef13c73-8951-4209-b0e0-c4df90d6a7ef None admin] RBAC: Authorizing `identity:get_project(project_id=96270b49feb741de94c788f1d9e65eb3)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:57.693341 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef13c73-8951-4209-b0e0-c4df90d6a7ef None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:57.694968 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 601/1191] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:07:57 2022] GET /identity/v3/projects/96270b49feb741de94c788f1d9e65eb3 => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:07:57.701204 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328d8f7-f0e6-4634-bfd7-830fc9be54fc None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:07:57.743646 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:07:57.804026 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328d8f7-f0e6-4634-bfd7-830fc9be54fc None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:07:57.836615 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507dcfac-9aa0-49a9-bb38-b4120646a014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:07:57.837258 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dcfac-9aa0-49a9-bb38-b4120646a014 None admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:57.837532 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dcfac-9aa0-49a9-bb38-b4120646a014 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:57.837799 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dcfac-9aa0-49a9-bb38-b4120646a014 None admin] PATH_INFO: `/v3/projects/96270b49feb741de94c788f1d9e65eb3/users/2590744c87dc4895ad1c14a48166f969/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:57.839681 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507dcfac-9aa0-49a9-bb38-b4120646a014 None admin] RBAC: Authorizing `identity:create_grant(project_id=96270b49feb741de94c788f1d9e65eb3, user_id=2590744c87dc4895ad1c14a48166f969, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:07:57.841585 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507dcfac-9aa0-49a9-bb38-b4120646a014 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:07:57.853623 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 591/1192] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Sep 5 12:07:57 2022] PUT /identity/v3/projects/96270b49feb741de94c788f1d9e65eb3/users/2590744c87dc4895ad1c14a48166f969/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:07:59.736132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87fd87-a6dd-497b-a322-b1b2819abff3 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:59.736660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87fd87-a6dd-497b-a322-b1b2819abff3 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:59.736660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87fd87-a6dd-497b-a322-b1b2819abff3 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:59.737935 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 602/1193] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:07:59.743495 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca8f1d-250c-4fc3-b5a8-f52df5bc3935 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:07:59.743958 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca8f1d-250c-4fc3-b5a8-f52df5bc3935 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:07:59.744056 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca8f1d-250c-4fc3-b5a8-f52df5bc3935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:07:59.764366 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-05ca8f1d-250c-4fc3-b5a8-f52df5bc3935 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:07:59.765575 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-05ca8f1d-250c-4fc3-b5a8-f52df5bc3935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:07:59.833479 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 592/1194] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:07:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:08:00.099180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87fd87-a6dd-497b-a322-b1b2819abff3 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:00.215540 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87fd87-a6dd-497b-a322-b1b2819abff3 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:00.236412 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:00.257959 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a51060-0012-4239-9865-e6512e749203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:00.258586 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a51060-0012-4239-9865-e6512e749203 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:00.258862 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a51060-0012-4239-9865-e6512e749203 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:00.259133 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a51060-0012-4239-9865-e6512e749203 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:00.269510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a51060-0012-4239-9865-e6512e749203 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:08:00.274208 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-16a51060-0012-4239-9865-e6512e749203 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:08:00.274208 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:08:00.274208 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:08:00.274208 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:08:00.274208 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:08:00.274208 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:08:00.274208 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:08:00.274208 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:08:00.274208 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:08:00.274208 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:08:00.274208 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:08:00.274208 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:08:00.274208 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:08:00.274208 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:08:00.274208 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:08:00.274208 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:08:00.275413 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:08:00.275413 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:08:00.275413 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:08:00.275413 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:08:00.275413 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:08:00.275413 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:08:00.275413 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:08:00.275413 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:08:00.275413 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:08:00.275413 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:08:00.275413 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:08:00.275413 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:08:00.275413 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:08:00.275413 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:08:00.275413 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:08:00.275413 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:08:00.276414 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:08:00.276414 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:08:00.276414 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:08:00.276414 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:08:00.276414 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:08:00.276414 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:08:00.276414 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:08:00.276414 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:08:00.276414 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:08:00.276414 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:08:00.276414 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:08:00.276414 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:08:00.276414 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 603/1195] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:08:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:08:00.282275 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca8f1d-250c-4fc3-b5a8-f52df5bc3935 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:00.313898 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:00.385819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca8f1d-250c-4fc3-b5a8-f52df5bc3935 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:00.425949 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b7a721-d513-474c-8f2e-c7f0987b9e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:00.426590 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7a721-d513-474c-8f2e-c7f0987b9e7e None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:00.426965 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7a721-d513-474c-8f2e-c7f0987b9e7e None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:00.427307 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7a721-d513-474c-8f2e-c7f0987b9e7e None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:00.428127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b7a721-d513-474c-8f2e-c7f0987b9e7e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:08:00.428461 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b7a721-d513-474c-8f2e-c7f0987b9e7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:00.430018 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b7a721-d513-474c-8f2e-c7f0987b9e7e None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:00.439349 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 593/1196] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:08:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:00.447825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a51060-0012-4239-9865-e6512e749203 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:00.561189 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a51060-0012-4239-9865-e6512e749203 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:00.596129 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108277db-add9-4f13-a0d8-b0bf6ef7f35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:00.596760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108277db-add9-4f13-a0d8-b0bf6ef7f35c None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:00.597034 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108277db-add9-4f13-a0d8-b0bf6ef7f35c None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:00.597305 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108277db-add9-4f13-a0d8-b0bf6ef7f35c None admin] PATH_INFO: `/v3/users/2590744c87dc4895ad1c14a48166f969` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:00.598437 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108277db-add9-4f13-a0d8-b0bf6ef7f35c None admin] RBAC: Authorizing `identity:get_user(user_id=2590744c87dc4895ad1c14a48166f969)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:00.599905 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108277db-add9-4f13-a0d8-b0bf6ef7f35c None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:00.601504 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 604/1197] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Sep 5 12:08:00 2022] GET /identity/v3/users/2590744c87dc4895ad1c14a48166f969 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:00.608219 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b7a721-d513-474c-8f2e-c7f0987b9e7e None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:00.708638 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:00.718419 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b7a721-d513-474c-8f2e-c7f0987b9e7e None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:00.756019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cfec4a-284a-4bed-a1ac-e9c4ce470857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:00.756737 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfec4a-284a-4bed-a1ac-e9c4ce470857 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:00.756967 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfec4a-284a-4bed-a1ac-e9c4ce470857 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:00.757177 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfec4a-284a-4bed-a1ac-e9c4ce470857 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:00.827628 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cfec4a-284a-4bed-a1ac-e9c4ce470857 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:00.829078 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cfec4a-284a-4bed-a1ac-e9c4ce470857 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:00.928026 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:00.941731 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 594/1198] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:08:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:08:00.947882 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108277db-add9-4f13-a0d8-b0bf6ef7f35c None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:00.998979 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:01.052895 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108277db-add9-4f13-a0d8-b0bf6ef7f35c None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:01.086862 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521e307-b45c-4e62-be34-d85b32862d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:01.087495 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521e307-b45c-4e62-be34-d85b32862d5b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:01.087808 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521e307-b45c-4e62-be34-d85b32862d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:01.088082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521e307-b45c-4e62-be34-d85b32862d5b None admin] PATH_INFO: `/v3/projects/96270b49feb741de94c788f1d9e65eb3` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:01.089195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1521e307-b45c-4e62-be34-d85b32862d5b None admin] RBAC: Authorizing `identity:get_project(project_id=96270b49feb741de94c788f1d9e65eb3)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:01.091293 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1521e307-b45c-4e62-be34-d85b32862d5b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:01.092977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 605/1199] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Sep 5 12:08:00 2022] GET /identity/v3/projects/96270b49feb741de94c788f1d9e65eb3 => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:01.099285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cfec4a-284a-4bed-a1ac-e9c4ce470857 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:01.201527 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cfec4a-284a-4bed-a1ac-e9c4ce470857 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:01.236071 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e2530-bf98-49d7-a3dd-5092e374afdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:01.236664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e2530-bf98-49d7-a3dd-5092e374afdd None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:01.237052 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e2530-bf98-49d7-a3dd-5092e374afdd None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:01.237319 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e2530-bf98-49d7-a3dd-5092e374afdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:01.238043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3e2530-bf98-49d7-a3dd-5092e374afdd None admin] RBAC: Adding query filter params (role.id=618c48706f7f445abeddf7eb2f8e927a, scope.project.id=96270b49feb741de94c788f1d9e65eb3, user.id=2590744c87dc4895ad1c14a48166f969) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:08:01.238344 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3e2530-bf98-49d7-a3dd-5092e374afdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:01.239779 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3e2530-bf98-49d7-a3dd-5092e374afdd None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:01.245876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 595/1200] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:08:01 2022] GET /identity/v3/role_assignments?user.id=2590744c87dc4895ad1c14a48166f969&scope.project.id=96270b49feb741de94c788f1d9e65eb3&role.id=618c48706f7f445abeddf7eb2f8e927a => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:11.082577 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=42 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:11.243135 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=72 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:58.057324 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f50dd-9098-44f4-bdb0-3bd65aa28f22 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:58.057935 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f50dd-9098-44f4-bdb0-3bd65aa28f22 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:58.057935 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f50dd-9098-44f4-bdb0-3bd65aa28f22 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:58.059643 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 606/1201] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:08:58.067573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853d415-3fbb-4bdd-bb9f-53aad82ca24d None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:58.067573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853d415-3fbb-4bdd-bb9f-53aad82ca24d None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:58.067573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853d415-3fbb-4bdd-bb9f-53aad82ca24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:58.087101 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-0853d415-3fbb-4bdd-bb9f-53aad82ca24d None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:08:58.088604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-0853d415-3fbb-4bdd-bb9f-53aad82ca24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:08:58.189531 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 596/1202] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:08:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:08:58.469336 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f50dd-9098-44f4-bdb0-3bd65aa28f22 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:58.619816 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f50dd-9098-44f4-bdb0-3bd65aa28f22 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:58.693790 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84123eed-9c1a-4506-bbda-8160b816cee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:58.694896 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84123eed-9c1a-4506-bbda-8160b816cee0 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:58.695283 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84123eed-9c1a-4506-bbda-8160b816cee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:58.695839 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84123eed-9c1a-4506-bbda-8160b816cee0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:58.715098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84123eed-9c1a-4506-bbda-8160b816cee0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:58.717807 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84123eed-9c1a-4506-bbda-8160b816cee0 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:58.748859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-84123eed-9c1a-4506-bbda-8160b816cee0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:08:58.748859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:08:58.748859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:08:58.748859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:08:58.748859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:08:58.748859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:08:58.748859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:08:58.748859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:08:58.748859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:08:58.748859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:08:58.748859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:08:58.748859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:08:58.748859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:08:58.748859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:08:58.748859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:08:58.748859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:08:58.749970 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:08:58.749970 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:08:58.749970 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:08:58.749970 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:08:58.749970 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:08:58.749970 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:08:58.749970 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:08:58.749970 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:08:58.749970 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:08:58.749970 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:08:58.749970 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:08:58.749970 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:08:58.749970 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:08:58.749970 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:08:58.749970 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:08:58.749970 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:08:58.749970 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:08:58.750966 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:08:58.750966 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:08:58.750966 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:08:58.751668 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 607/1203] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:08:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:08:58.756950 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0853d415-3fbb-4bdd-bb9f-53aad82ca24d None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:58.889996 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0853d415-3fbb-4bdd-bb9f-53aad82ca24d None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:58.932689 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:58.940309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee6848-268d-4145-a600-ab48c2257ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:58.940976 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee6848-268d-4145-a600-ab48c2257ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:58.941255 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee6848-268d-4145-a600-ab48c2257ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:58.941525 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee6848-268d-4145-a600-ab48c2257ee9 None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:58.942260 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee6848-268d-4145-a600-ab48c2257ee9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:08:58.942634 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee6848-268d-4145-a600-ab48c2257ee9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:58.944233 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee6848-268d-4145-a600-ab48c2257ee9 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:58.967010 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 597/1204] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:08:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:58.973936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84123eed-9c1a-4506-bbda-8160b816cee0 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:59.042560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:59.109040 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84123eed-9c1a-4506-bbda-8160b816cee0 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:59.156353 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6e7b1-5286-47c8-bc01-f7710c237f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:59.157157 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6e7b1-5286-47c8-bc01-f7710c237f69 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:59.157549 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6e7b1-5286-47c8-bc01-f7710c237f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:59.157968 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6e7b1-5286-47c8-bc01-f7710c237f69 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:59.158779 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6e7b1-5286-47c8-bc01-f7710c237f69 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:59.160750 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6e7b1-5286-47c8-bc01-f7710c237f69 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:59.211694 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 608/1205] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Sep 5 12:08:58 2022] POST /identity/v3/users => generated 251 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:09:01.536365 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a56497-a707-408e-999d-4dcc407b0d15 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:01.537038 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a56497-a707-408e-999d-4dcc407b0d15 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:01.537038 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a56497-a707-408e-999d-4dcc407b0d15 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:01.538736 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 598/1206] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:01.547458 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb8322-8d79-4107-8c55-b11128784779 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:01.548052 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb8322-8d79-4107-8c55-b11128784779 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:01.548052 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb8322-8d79-4107-8c55-b11128784779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:01.569508 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-9cdb8322-8d79-4107-8c55-b11128784779 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:01.570824 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9cdb8322-8d79-4107-8c55-b11128784779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:01.648158 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 609/1207] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:09:01.930544 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a56497-a707-408e-999d-4dcc407b0d15 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:02.059434 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a56497-a707-408e-999d-4dcc407b0d15 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:02.111544 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820da13-8b1d-4657-bde3-b958596224de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:02.112189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820da13-8b1d-4657-bde3-b958596224de None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:02.112460 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820da13-8b1d-4657-bde3-b958596224de None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:02.112738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820da13-8b1d-4657-bde3-b958596224de None admin] PATH_INFO: `/v3/roles/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:02.127160 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5820da13-8b1d-4657-bde3-b958596224de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:02.132168 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-5820da13-8b1d-4657-bde3-b958596224de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:09:02.132168 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:02.132168 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:02.132168 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:02.132168 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:02.132168 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:02.132168 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:02.132168 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:02.132168 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:02.132168 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:02.132168 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:02.132168 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:02.132168 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:02.132168 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:02.132168 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:02.132168 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:02.133306 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:02.133306 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:02.133306 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:02.133306 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:02.133306 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:02.133306 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:02.133306 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:02.133306 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:02.133306 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:02.133306 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:02.133306 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:02.133306 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:02.133306 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:02.133306 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:02.133306 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:02.133306 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:02.134629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:02.134629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:02.134629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:02.134629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:02.134629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:02.134629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:02.134629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:02.134629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:02.134629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:02.134629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:02.134629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:09:02.134629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:09:02.134629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 599/1208] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Sep 5 12:09:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:09:02.140367 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb8322-8d79-4107-8c55-b11128784779 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:02.186266 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:02.301854 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb8322-8d79-4107-8c55-b11128784779 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:02.352406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d698d8-5a23-45e7-ae9c-d3fff251eabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:02.352992 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d698d8-5a23-45e7-ae9c-d3fff251eabe None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:02.353340 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d698d8-5a23-45e7-ae9c-d3fff251eabe None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:02.353606 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d698d8-5a23-45e7-ae9c-d3fff251eabe None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:02.354310 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d698d8-5a23-45e7-ae9c-d3fff251eabe None admin] RBAC: Adding query filter params (name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:02.354659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d698d8-5a23-45e7-ae9c-d3fff251eabe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:02.356237 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d698d8-5a23-45e7-ae9c-d3fff251eabe None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:02.372139 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 610/1209] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Sep 5 12:09:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:02.378043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820da13-8b1d-4657-bde3-b958596224de None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:02.456230 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:02.536720 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820da13-8b1d-4657-bde3-b958596224de None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:02.596000 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8374f65e-c427-443b-bbd5-51b9f34f6551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:02.596992 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374f65e-c427-443b-bbd5-51b9f34f6551 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:02.597455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374f65e-c427-443b-bbd5-51b9f34f6551 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:02.597850 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374f65e-c427-443b-bbd5-51b9f34f6551 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:02.610112 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8374f65e-c427-443b-bbd5-51b9f34f6551 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:02.611590 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8374f65e-c427-443b-bbd5-51b9f34f6551 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:02.628885 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-8374f65e-c427-443b-bbd5-51b9f34f6551 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:02.628885 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:02.628885 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:02.628885 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:02.628885 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:02.628885 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:02.628885 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:02.628885 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:02.628885 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:02.628885 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:02.628885 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:02.628885 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:02.628885 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:02.628885 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:02.628885 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:02.628885 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:02.629952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:02.629952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:02.629952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:02.629952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:02.629952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:02.629952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:02.629952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:02.629952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:02.629952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:02.629952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:02.629952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:02.629952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:02.629952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:02.629952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:02.629952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:02.629952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:02.629952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:02.630909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:02.630909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:02.630909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:09:02.630909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 600/1210] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:02.636470 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d698d8-5a23-45e7-ae9c-d3fff251eabe None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:02.766880 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:02.769745 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d698d8-5a23-45e7-ae9c-d3fff251eabe None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:02.830336 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc01a1-dea7-4740-aa56-269eee5bfa38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:02.831097 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc01a1-dea7-4740-aa56-269eee5bfa38 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:02.831568 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc01a1-dea7-4740-aa56-269eee5bfa38 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:02.832044 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc01a1-dea7-4740-aa56-269eee5bfa38 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:02.832872 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc01a1-dea7-4740-aa56-269eee5bfa38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:02.833277 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc01a1-dea7-4740-aa56-269eee5bfa38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:02.835334 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc01a1-dea7-4740-aa56-269eee5bfa38 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:02.860965 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 611/1211] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:02.862823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8374f65e-c427-443b-bbd5-51b9f34f6551 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:02.995898 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8374f65e-c427-443b-bbd5-51b9f34f6551 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:02.997887 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:03.035307 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee473e-54bc-4e6f-90bd-514269fd2c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:03.036470 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee473e-54bc-4e6f-90bd-514269fd2c62 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:03.036470 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee473e-54bc-4e6f-90bd-514269fd2c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:03.036688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee473e-54bc-4e6f-90bd-514269fd2c62 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:03.046395 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ee473e-54bc-4e6f-90bd-514269fd2c62 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:03.047902 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ee473e-54bc-4e6f-90bd-514269fd2c62 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:03.060478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-99ee473e-54bc-4e6f-90bd-514269fd2c62 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 05 12:09:03.060478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:03.060478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:03.060478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:03.060478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:03.060478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:03.060478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:03.060478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:03.060478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:03.060478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:03.060478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:03.060478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:03.060478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:09:03.060478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:09:03.060478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:09:03.060478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:09:03.061565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:03.061565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:03.061565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:09:03.061565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:09:03.061565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:09:03.061565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:09:03.061565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:03.061565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:03.061565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:03.061565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:03.061565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:03.061565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:03.061565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:03.061565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:03.061565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:03.061565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:03.061565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:03.061565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:03.063112 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:03.063112 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:03.063112 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:09:03.063112 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:09:03.063112 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:09:03.063112 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:09:03.063112 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:09:03.063112 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:09:03.063112 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 05 12:09:03.063112 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:09:03.063112 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 601/1212] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:03.067883 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc01a1-dea7-4740-aa56-269eee5bfa38 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:03.210577 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc01a1-dea7-4740-aa56-269eee5bfa38 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:03.258632 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b6262-2c6b-4ca9-ada1-ded59e543e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:03.259622 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b6262-2c6b-4ca9-ada1-ded59e543e90 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:03.260070 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b6262-2c6b-4ca9-ada1-ded59e543e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:03.260364 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b6262-2c6b-4ca9-ada1-ded59e543e90 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:03.261091 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3b6262-2c6b-4ca9-ada1-ded59e543e90 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:03.261435 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3b6262-2c6b-4ca9-ada1-ded59e543e90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:03.262942 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3b6262-2c6b-4ca9-ada1-ded59e543e90 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:03.300484 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 612/1213] 10.222.0.16 () {56 vars in 1143 bytes} [Mon Sep 5 12:09:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:03.306359 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee473e-54bc-4e6f-90bd-514269fd2c62 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:03.468018 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee473e-54bc-4e6f-90bd-514269fd2c62 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:03.547052 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313874d4-21e8-4a38-b829-0938ec3e6f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:03.548606 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313874d4-21e8-4a38-b829-0938ec3e6f68 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:03.549170 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313874d4-21e8-4a38-b829-0938ec3e6f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:03.549904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313874d4-21e8-4a38-b829-0938ec3e6f68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:03.612538 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:03.670396 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313874d4-21e8-4a38-b829-0938ec3e6f68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:03.673847 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313874d4-21e8-4a38-b829-0938ec3e6f68 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:03.817773 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 602/1214] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:09:03.819294 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b6262-2c6b-4ca9-ada1-ded59e543e90 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:03.847874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:03.977478 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b6262-2c6b-4ca9-ada1-ded59e543e90 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:04.032670 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec3141-6e7b-4876-b5af-33205933eef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:04.033593 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec3141-6e7b-4876-b5af-33205933eef1 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:04.033908 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec3141-6e7b-4876-b5af-33205933eef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:04.034229 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec3141-6e7b-4876-b5af-33205933eef1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:04.047141 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec3141-6e7b-4876-b5af-33205933eef1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:04.048823 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec3141-6e7b-4876-b5af-33205933eef1 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:04.074350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-fcec3141-6e7b-4876-b5af-33205933eef1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:04.074350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:04.074350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:04.074350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:04.074350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:04.074350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:04.074350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:04.074350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:04.074350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:04.074350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:04.074350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:04.074350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:04.074350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:04.074350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:04.074350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:04.074350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:04.075436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:04.075436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:04.075436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:04.075436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:04.075436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:04.075436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:04.075436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:04.075436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:04.075436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:04.075436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:04.075436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:04.075436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:04.075436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:04.075436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:04.075436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:04.075436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:04.075436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:04.076418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:04.076418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:04.076418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:09:04.076418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 613/1215] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:04.082593 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313874d4-21e8-4a38-b829-0938ec3e6f68 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:04.168750 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:04.220652 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313874d4-21e8-4a38-b829-0938ec3e6f68 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:04.275287 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d806cfa-4b13-43eb-baae-4b43b7f47756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:04.276117 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d806cfa-4b13-43eb-baae-4b43b7f47756 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:04.276428 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d806cfa-4b13-43eb-baae-4b43b7f47756 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:04.276734 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d806cfa-4b13-43eb-baae-4b43b7f47756 None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:04.278105 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d806cfa-4b13-43eb-baae-4b43b7f47756 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:04.278531 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d806cfa-4b13-43eb-baae-4b43b7f47756 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:04.280334 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d806cfa-4b13-43eb-baae-4b43b7f47756 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:04.298352 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 603/1216] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:09:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:04.305219 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec3141-6e7b-4876-b5af-33205933eef1 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:04.398171 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:04.451598 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec3141-6e7b-4876-b5af-33205933eef1 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:04.514813 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2147cd8d-5547-447d-80b5-e907caf46347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:04.515539 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147cd8d-5547-447d-80b5-e907caf46347 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:04.517484 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147cd8d-5547-447d-80b5-e907caf46347 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:04.517484 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147cd8d-5547-447d-80b5-e907caf46347 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:04.527721 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2147cd8d-5547-447d-80b5-e907caf46347 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:04.528308 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2147cd8d-5547-447d-80b5-e907caf46347 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:04.544635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-2147cd8d-5547-447d-80b5-e907caf46347 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:09:04.544635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:04.544635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:04.544635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:04.544635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:04.544635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:04.544635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:04.544635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:04.544635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:04.544635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:04.544635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:04.544635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:04.544635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:09:04.544635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:09:04.544635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:09:04.546074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:09:04.546074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:04.546074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:04.546074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:04.546074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:04.546074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:04.546074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:04.546074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:04.546074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:04.546074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:04.546074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:04.546074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:04.546074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:04.546074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:04.546074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:04.546074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:04.546074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:04.547061 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:09:04.547061 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:09:04.547061 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:09:04.547061 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:09:04.547061 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:09:04.547061 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:09:04.547061 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:09:04.547061 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:09:04.547061 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 614/1217] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:09:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:04.552768 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d806cfa-4b13-43eb-baae-4b43b7f47756 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:04.702519 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d806cfa-4b13-43eb-baae-4b43b7f47756 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:04.730707 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:04.750192 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f1118f-7011-4748-b077-10f3f9418b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:04.751729 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1118f-7011-4748-b077-10f3f9418b0c None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:04.752206 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1118f-7011-4748-b077-10f3f9418b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:04.752679 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1118f-7011-4748-b077-10f3f9418b0c None admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:04.753941 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f1118f-7011-4748-b077-10f3f9418b0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:04.754474 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f1118f-7011-4748-b077-10f3f9418b0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:04.757219 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f1118f-7011-4748-b077-10f3f9418b0c None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:04.784041 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 604/1218] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:09:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:04.790425 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2147cd8d-5547-447d-80b5-e907caf46347 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:04.940685 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2147cd8d-5547-447d-80b5-e907caf46347 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:04.973040 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:04.984292 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad7d14-fc9d-44f5-85f8-19215f91cc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:04.986666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad7d14-fc9d-44f5-85f8-19215f91cc43 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:04.986666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad7d14-fc9d-44f5-85f8-19215f91cc43 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:04.986666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad7d14-fc9d-44f5-85f8-19215f91cc43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:04.987592 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad7d14-fc9d-44f5-85f8-19215f91cc43 None admin] RBAC: Adding query filter params (role.id=8cf764a0144a490a822fcf86d7aceeb0, scope.project.id=fa3178684d184ec880f17640d19d55b7, user.id=524bcfffcdfa4790885e0d4cbd66fa80) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:04.988166 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad7d14-fc9d-44f5-85f8-19215f91cc43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:04.990229 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad7d14-fc9d-44f5-85f8-19215f91cc43 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:05.000152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 615/1219] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:09:04 2022] GET /identity/v3/role_assignments?user.id=524bcfffcdfa4790885e0d4cbd66fa80&scope.project.id=fa3178684d184ec880f17640d19d55b7&role.id=8cf764a0144a490a822fcf86d7aceeb0 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:07.017639 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256003a-d811-456f-8d97-a634609468b2 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:07.018194 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256003a-d811-456f-8d97-a634609468b2 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:07.018278 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256003a-d811-456f-8d97-a634609468b2 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:07.019761 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 605/1220] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:09:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:07.025263 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42612af6-c797-492b-a4bd-de2fe1eae151 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:07.025701 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42612af6-c797-492b-a4bd-de2fe1eae151 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:07.025816 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42612af6-c797-492b-a4bd-de2fe1eae151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:07.046338 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-42612af6-c797-492b-a4bd-de2fe1eae151 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:07.047703 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-42612af6-c797-492b-a4bd-de2fe1eae151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:07.128557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 616/1221] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:09:07.418298 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256003a-d811-456f-8d97-a634609468b2 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:07.540433 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256003a-d811-456f-8d97-a634609468b2 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:07.580757 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d034234-baaa-4297-85ea-85ee2b102604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:07.581393 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d034234-baaa-4297-85ea-85ee2b102604 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:07.581685 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d034234-baaa-4297-85ea-85ee2b102604 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:07.581940 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d034234-baaa-4297-85ea-85ee2b102604 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:07.592753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d034234-baaa-4297-85ea-85ee2b102604 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:07.597673 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-9d034234-baaa-4297-85ea-85ee2b102604 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:09:07.597673 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:07.597673 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:07.597673 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:07.597673 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:07.597673 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:07.597673 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:07.597673 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:07.597673 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:07.597673 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:07.597673 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:07.597673 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:07.597673 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:07.597673 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:07.597673 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:07.597673 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:07.598683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:07.598683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:07.598683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:07.598683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:07.598683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:07.598683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:07.598683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:07.598683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:07.598683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:07.598683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:07.598683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:07.598683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:07.598683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:07.598683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:07.598683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:07.598683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:07.599695 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:07.599695 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:07.599695 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:07.599695 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:07.599695 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:07.599695 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:07.599695 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:07.599695 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:07.599695 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:07.599695 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:07.599695 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:09:07.599695 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:09:07.599695 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 606/1222] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Sep 5 12:09:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:07.605180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42612af6-c797-492b-a4bd-de2fe1eae151 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:07.740123 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42612af6-c797-492b-a4bd-de2fe1eae151 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:07.769971 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:07.783816 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0943f80-bb5b-404f-8926-005f9a6a7ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:07.784435 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0943f80-bb5b-404f-8926-005f9a6a7ede None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:07.784721 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0943f80-bb5b-404f-8926-005f9a6a7ede None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:07.784999 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0943f80-bb5b-404f-8926-005f9a6a7ede None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:07.785671 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0943f80-bb5b-404f-8926-005f9a6a7ede None admin] RBAC: Adding query filter params (name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:07.785975 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0943f80-bb5b-404f-8926-005f9a6a7ede None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:07.787377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0943f80-bb5b-404f-8926-005f9a6a7ede None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:07.797345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 617/1223] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Sep 5 12:09:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:07.806341 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d034234-baaa-4297-85ea-85ee2b102604 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:07.853593 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:07.920435 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d034234-baaa-4297-85ea-85ee2b102604 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:07.959928 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4419deec-7907-4909-aa7e-fbe182ec1383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:07.960637 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419deec-7907-4909-aa7e-fbe182ec1383 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:07.960935 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419deec-7907-4909-aa7e-fbe182ec1383 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:07.961275 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419deec-7907-4909-aa7e-fbe182ec1383 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:07.973759 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4419deec-7907-4909-aa7e-fbe182ec1383 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:07.975265 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4419deec-7907-4909-aa7e-fbe182ec1383 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:07.990618 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-4419deec-7907-4909-aa7e-fbe182ec1383 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:07.990618 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:07.990618 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:07.990618 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:07.990618 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:07.990618 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:07.990618 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:07.990618 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:07.990618 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:07.990618 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:07.990618 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:07.990618 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:07.990618 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:07.990618 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:07.990618 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:07.990618 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:07.992033 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:07.992033 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:07.992033 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:07.992033 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:07.992033 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:07.992033 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:07.992033 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:07.992033 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:07.992033 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:07.992033 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:07.992033 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:07.992033 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:07.992033 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:07.992033 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:07.992033 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:07.992033 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:07.992033 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:07.993074 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:07.993074 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:07.993074 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:09:07.996417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 607/1224] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:07.998226 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0943f80-bb5b-404f-8926-005f9a6a7ede None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:08.111951 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0943f80-bb5b-404f-8926-005f9a6a7ede None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:08.147235 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8cf98-0918-47a7-9fdd-bd1e52f72c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:08.147878 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8cf98-0918-47a7-9fdd-bd1e52f72c85 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:08.148224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8cf98-0918-47a7-9fdd-bd1e52f72c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:08.148435 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8cf98-0918-47a7-9fdd-bd1e52f72c85 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:08.149080 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8cf98-0918-47a7-9fdd-bd1e52f72c85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:08.149384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8cf98-0918-47a7-9fdd-bd1e52f72c85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:08.150749 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8cf98-0918-47a7-9fdd-bd1e52f72c85 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:08.163760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 618/1225] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:08.170238 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4419deec-7907-4909-aa7e-fbe182ec1383 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:08.272782 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:08.280442 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4419deec-7907-4909-aa7e-fbe182ec1383 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:08.316835 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ed0a9-f87c-4233-9f19-d88f36e0b00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:08.317374 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ed0a9-f87c-4233-9f19-d88f36e0b00b None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:08.317650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ed0a9-f87c-4233-9f19-d88f36e0b00b None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:08.317920 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ed0a9-f87c-4233-9f19-d88f36e0b00b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:08.325853 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ed0a9-f87c-4233-9f19-d88f36e0b00b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:08.327277 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ed0a9-f87c-4233-9f19-d88f36e0b00b None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:08.339043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-975ed0a9-f87c-4233-9f19-d88f36e0b00b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 05 12:09:08.339043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:08.339043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:08.339043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:08.339043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:08.339043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:08.339043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:08.339043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:08.339043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:08.339043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:08.339043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:08.339043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:08.339043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:09:08.339043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:09:08.339043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:09:08.339043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:09:08.340809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:08.340809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:08.340809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:09:08.340809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:09:08.340809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:09:08.340809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:09:08.340809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:08.340809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:08.340809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:08.340809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:08.340809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:08.340809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:08.340809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:08.340809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:08.340809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:08.340809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:08.340809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:08.340809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:08.341792 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:08.341792 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:08.341792 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:09:08.341792 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:09:08.341792 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:09:08.341792 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:09:08.341792 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:09:08.341792 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:09:08.341792 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 05 12:09:08.341792 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:09:08.341792 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 608/1226] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:08.346626 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8cf98-0918-47a7-9fdd-bd1e52f72c85 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:08.402587 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:08.463947 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8cf98-0918-47a7-9fdd-bd1e52f72c85 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:08.504348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda143c-54a9-4f19-bc39-070274dea28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:08.505283 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda143c-54a9-4f19-bc39-070274dea28a None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:08.505804 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda143c-54a9-4f19-bc39-070274dea28a None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:08.506215 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda143c-54a9-4f19-bc39-070274dea28a None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:08.507165 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda143c-54a9-4f19-bc39-070274dea28a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:08.507619 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda143c-54a9-4f19-bc39-070274dea28a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:08.509557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda143c-54a9-4f19-bc39-070274dea28a None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:08.536372 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 619/1227] 10.222.0.16 () {56 vars in 1143 bytes} [Mon Sep 5 12:09:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:08.542671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ed0a9-f87c-4233-9f19-d88f36e0b00b None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:08.658196 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ed0a9-f87c-4233-9f19-d88f36e0b00b None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:08.700385 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779cebcb-d30f-4990-b915-b424a1f39d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:08.701057 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779cebcb-d30f-4990-b915-b424a1f39d5f None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:08.701334 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779cebcb-d30f-4990-b915-b424a1f39d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:08.701731 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779cebcb-d30f-4990-b915-b424a1f39d5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:08.714164 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779cebcb-d30f-4990-b915-b424a1f39d5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:08.716566 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779cebcb-d30f-4990-b915-b424a1f39d5f None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:08.732908 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-779cebcb-d30f-4990-b915-b424a1f39d5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:08.732908 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:08.732908 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:08.732908 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:08.732908 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:08.732908 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:08.732908 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:08.732908 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:08.732908 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:08.732908 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:08.732908 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:08.732908 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:08.732908 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:08.732908 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:08.732908 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:08.732908 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:08.734189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:08.734189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:08.734189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:08.734189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:08.734189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:08.734189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:08.734189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:08.734189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:08.734189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:08.734189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:08.734189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:08.734189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:08.734189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:08.734189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:08.734189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:08.734189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:08.734189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:08.735143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:08.735143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:08.735143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:09:08.735143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 609/1228] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:08.740238 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda143c-54a9-4f19-bc39-070274dea28a None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:08.854614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda143c-54a9-4f19-bc39-070274dea28a None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:08.857117 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:08.895878 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfa1c0-5382-4e83-915e-67bbd770635c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:08.896586 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfa1c0-5382-4e83-915e-67bbd770635c None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:08.896921 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfa1c0-5382-4e83-915e-67bbd770635c None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:08.897222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfa1c0-5382-4e83-915e-67bbd770635c None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:08.898030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dfa1c0-5382-4e83-915e-67bbd770635c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:08.898467 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dfa1c0-5382-4e83-915e-67bbd770635c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:08.900077 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dfa1c0-5382-4e83-915e-67bbd770635c None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:08.913326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 620/1229] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:09:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:08.919272 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779cebcb-d30f-4990-b915-b424a1f39d5f None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:08.931271 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:09.047575 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779cebcb-d30f-4990-b915-b424a1f39d5f None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:09.088958 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35a006-b897-4c5a-b443-264472886f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:09.089741 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35a006-b897-4c5a-b443-264472886f07 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:09.090028 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35a006-b897-4c5a-b443-264472886f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:09.090379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35a006-b897-4c5a-b443-264472886f07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:09.098954 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa35a006-b897-4c5a-b443-264472886f07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:09.100600 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa35a006-b897-4c5a-b443-264472886f07 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:09.113636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-aa35a006-b897-4c5a-b443-264472886f07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:09:09.113636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:09.113636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:09.113636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:09.113636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:09.113636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:09.113636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:09.113636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:09.113636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:09.113636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:09.113636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:09.113636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:09.113636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:09:09.113636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:09:09.113636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:09:09.114676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:09:09.114676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:09.114676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:09.114676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:09.114676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:09.114676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:09.114676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:09.114676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:09.114676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:09.114676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:09.114676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:09.114676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:09.114676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:09.114676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:09.114676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:09.114676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:09.114676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:09.116134 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:09:09.116134 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:09:09.116134 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:09:09.116134 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:09:09.116134 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:09:09.116134 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:09:09.116134 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:09:09.116134 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:09:09.116134 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 610/1230] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:09:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:09:09.122245 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfa1c0-5382-4e83-915e-67bbd770635c None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:09.248310 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfa1c0-5382-4e83-915e-67bbd770635c None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:09.282381 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a89b0-5075-45a4-b0a2-e8521680ac90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:09.282990 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a89b0-5075-45a4-b0a2-e8521680ac90 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:09.283262 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a89b0-5075-45a4-b0a2-e8521680ac90 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:09.283557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a89b0-5075-45a4-b0a2-e8521680ac90 None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:09.284251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0a89b0-5075-45a4-b0a2-e8521680ac90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:09.284553 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0a89b0-5075-45a4-b0a2-e8521680ac90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:09.285977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0a89b0-5075-45a4-b0a2-e8521680ac90 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:09.299601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 621/1231] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:09:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:09.306430 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35a006-b897-4c5a-b443-264472886f07 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:09.387362 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:09.426711 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35a006-b897-4c5a-b443-264472886f07 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:09.466932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a23755d-6e35-480b-93f5-8b869fee04ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:09.467720 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23755d-6e35-480b-93f5-8b869fee04ac None admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:09.467997 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23755d-6e35-480b-93f5-8b869fee04ac None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:09.468269 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23755d-6e35-480b-93f5-8b869fee04ac None admin] PATH_INFO: `/v3/projects/fa3178684d184ec880f17640d19d55b7/users/524bcfffcdfa4790885e0d4cbd66fa80/roles/8cf764a0144a490a822fcf86d7aceeb0` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:09.485773 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a23755d-6e35-480b-93f5-8b869fee04ac None admin] RBAC: Authorizing `identity:create_grant(project_id=fa3178684d184ec880f17640d19d55b7, user_id=524bcfffcdfa4790885e0d4cbd66fa80, role_id=8cf764a0144a490a822fcf86d7aceeb0)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:09.487246 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a23755d-6e35-480b-93f5-8b869fee04ac None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:09.502817 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 611/1232] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Sep 5 12:09:09 2022] PUT /identity/v3/projects/fa3178684d184ec880f17640d19d55b7/users/524bcfffcdfa4790885e0d4cbd66fa80/roles/8cf764a0144a490a822fcf86d7aceeb0 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:09:11.497313 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2e454-9315-466a-95d4-9cda05458f2f None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:11.497921 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2e454-9315-466a-95d4-9cda05458f2f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:11.497921 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2e454-9315-466a-95d4-9cda05458f2f None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:11.500625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 622/1233] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:09:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:11.506276 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f3c8e-9b9f-440a-9da6-4de21781ca5d None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:11.506764 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f3c8e-9b9f-440a-9da6-4de21781ca5d None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:11.506867 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f3c8e-9b9f-440a-9da6-4de21781ca5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:11.525416 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-b40f3c8e-9b9f-440a-9da6-4de21781ca5d None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:11.526597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-b40f3c8e-9b9f-440a-9da6-4de21781ca5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:11.603558 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 612/1234] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:09:11.885207 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d2e454-9315-466a-95d4-9cda05458f2f None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:11.912869 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:11.997273 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d2e454-9315-466a-95d4-9cda05458f2f None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:12.035250 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a2ce5-eb49-4064-b2f8-b998457e5b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:12.035924 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a2ce5-eb49-4064-b2f8-b998457e5b13 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:12.036204 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a2ce5-eb49-4064-b2f8-b998457e5b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:12.036480 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a2ce5-eb49-4064-b2f8-b998457e5b13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:12.046557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161a2ce5-eb49-4064-b2f8-b998457e5b13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:12.051268 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-161a2ce5-eb49-4064-b2f8-b998457e5b13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:09:12.051268 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:12.051268 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:12.051268 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:12.051268 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:12.051268 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:12.051268 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:12.051268 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:12.051268 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:12.051268 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:12.051268 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:12.051268 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:12.051268 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:12.051268 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:12.051268 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:12.051268 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:12.052547 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:12.052547 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:12.052547 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:12.052547 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:12.052547 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:12.052547 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:12.052547 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:12.052547 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:12.052547 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:12.052547 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:12.052547 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:12.052547 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:12.052547 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:12.052547 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:12.052547 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:12.052547 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:12.054034 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:12.054034 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:12.054034 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:12.054034 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:12.054034 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:12.054034 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:12.054034 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:12.054034 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:12.054034 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:12.054034 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:12.054034 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:09:12.054034 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:09:12.054034 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 623/1235] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Sep 5 12:09:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:12.059636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f3c8e-9b9f-440a-9da6-4de21781ca5d None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:12.115372 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:12.174345 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f3c8e-9b9f-440a-9da6-4de21781ca5d None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:12.213139 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00c1c0-0244-472b-8cbf-7add6027ae14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:12.213909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00c1c0-0244-472b-8cbf-7add6027ae14 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:12.214192 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00c1c0-0244-472b-8cbf-7add6027ae14 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:12.214497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00c1c0-0244-472b-8cbf-7add6027ae14 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:12.215148 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00c1c0-0244-472b-8cbf-7add6027ae14 None admin] RBAC: Adding query filter params (name=service) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:12.215460 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00c1c0-0244-472b-8cbf-7add6027ae14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:12.216851 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00c1c0-0244-472b-8cbf-7add6027ae14 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:12.226313 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 613/1236] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Sep 5 12:09:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:12.233550 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a2ce5-eb49-4064-b2f8-b998457e5b13 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:12.333783 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:12.356429 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a2ce5-eb49-4064-b2f8-b998457e5b13 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:12.400009 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f93adf-25b8-4e6c-b29c-8e4bf329f0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:12.400646 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f93adf-25b8-4e6c-b29c-8e4bf329f0b9 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:12.400929 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f93adf-25b8-4e6c-b29c-8e4bf329f0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:12.401203 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f93adf-25b8-4e6c-b29c-8e4bf329f0b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:12.413659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f93adf-25b8-4e6c-b29c-8e4bf329f0b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:12.415197 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f93adf-25b8-4e6c-b29c-8e4bf329f0b9 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:12.431372 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-c8f93adf-25b8-4e6c-b29c-8e4bf329f0b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:12.431372 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:12.431372 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:12.431372 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:12.431372 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:12.431372 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:12.431372 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:12.431372 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:12.431372 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:12.431372 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:12.431372 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:12.431372 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:12.431372 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:12.431372 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:12.431372 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:12.431372 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:12.432666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:12.432666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:12.432666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:12.432666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:12.432666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:12.432666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:12.432666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:12.432666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:12.432666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:12.432666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:12.432666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:12.432666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:12.432666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:12.432666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:12.432666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:12.432666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:12.432666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:12.434146 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:12.434146 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:12.434146 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:09:12.438391 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 624/1237] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:12.439994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00c1c0-0244-472b-8cbf-7add6027ae14 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:12.552547 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00c1c0-0244-472b-8cbf-7add6027ae14 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:12.562987 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:12.591854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6128fc-0ad8-4822-996a-7f90bd5a8b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:12.592637 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6128fc-0ad8-4822-996a-7f90bd5a8b93 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:12.592922 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6128fc-0ad8-4822-996a-7f90bd5a8b93 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:12.593195 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6128fc-0ad8-4822-996a-7f90bd5a8b93 None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:12.593900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6128fc-0ad8-4822-996a-7f90bd5a8b93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:12.594290 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6128fc-0ad8-4822-996a-7f90bd5a8b93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:12.595827 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6128fc-0ad8-4822-996a-7f90bd5a8b93 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:12.616406 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 614/1238] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:12.622483 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f93adf-25b8-4e6c-b29c-8e4bf329f0b9 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:12.753321 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f93adf-25b8-4e6c-b29c-8e4bf329f0b9 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:12.796012 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b0b086-ae61-4ac7-8e63-432dcb1138ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:12.796806 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0b086-ae61-4ac7-8e63-432dcb1138ee None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:12.797080 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0b086-ae61-4ac7-8e63-432dcb1138ee None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:12.797447 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0b086-ae61-4ac7-8e63-432dcb1138ee None admin] PATH_INFO: `/v3/users/placement` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:12.800653 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:12.807025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b0b086-ae61-4ac7-8e63-432dcb1138ee None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:12.808472 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b0b086-ae61-4ac7-8e63-432dcb1138ee None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:12.829978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-b6b0b086-ae61-4ac7-8e63-432dcb1138ee None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 05 12:09:12.829978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:12.829978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:12.829978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:12.829978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:12.829978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:12.829978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:12.829978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:12.829978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:12.829978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:12.829978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:12.829978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:12.829978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:09:12.829978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:09:12.829978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:09:12.829978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:09:12.831145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:12.831145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:12.831145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:09:12.831145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:09:12.831145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:09:12.831145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:09:12.831145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:12.831145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:12.831145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:12.831145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:12.831145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:12.831145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:12.831145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:12.831145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:12.831145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:12.831145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:12.831145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:12.831145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:12.837251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:12.837251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:12.837251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:09:12.837251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:09:12.837251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:09:12.837251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:09:12.837251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:09:12.837251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:09:12.837251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 05 12:09:12.837251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:09:12.837251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 625/1239] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:12.838002 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6128fc-0ad8-4822-996a-7f90bd5a8b93 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:12.954685 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6128fc-0ad8-4822-996a-7f90bd5a8b93 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:12.992792 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbaaa07-29c4-4c32-9661-3af413116c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:12.993390 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaaa07-29c4-4c32-9661-3af413116c7d None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:12.993663 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaaa07-29c4-4c32-9661-3af413116c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:12.993934 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaaa07-29c4-4c32-9661-3af413116c7d None admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:12.994738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbaaa07-29c4-4c32-9661-3af413116c7d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:12.995030 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbaaa07-29c4-4c32-9661-3af413116c7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:12.996626 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbaaa07-29c4-4c32-9661-3af413116c7d None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:13.015786 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 615/1240] 10.222.0.16 () {56 vars in 1143 bytes} [Mon Sep 5 12:09:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:13.021901 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b0b086-ae61-4ac7-8e63-432dcb1138ee None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:13.155281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b0b086-ae61-4ac7-8e63-432dcb1138ee None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:13.193100 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d53d9bb-7cdb-47a9-9367-c03762a2dcf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:13.193703 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53d9bb-7cdb-47a9-9367-c03762a2dcf4 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:13.193975 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53d9bb-7cdb-47a9-9367-c03762a2dcf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:13.194238 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53d9bb-7cdb-47a9-9367-c03762a2dcf4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:13.265585 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:13.273948 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d53d9bb-7cdb-47a9-9367-c03762a2dcf4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:13.275157 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d53d9bb-7cdb-47a9-9367-c03762a2dcf4 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:13.420305 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 626/1241] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 05 12:09:13.426465 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbaaa07-29c4-4c32-9661-3af413116c7d None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:13.457410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:13.542112 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbaaa07-29c4-4c32-9661-3af413116c7d None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:13.586546 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b592232-a678-42a9-9dc3-a7d4528dee32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:13.587297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b592232-a678-42a9-9dc3-a7d4528dee32 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:13.587656 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b592232-a678-42a9-9dc3-a7d4528dee32 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:13.587962 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b592232-a678-42a9-9dc3-a7d4528dee32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:13.599924 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b592232-a678-42a9-9dc3-a7d4528dee32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:13.601759 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b592232-a678-42a9-9dc3-a7d4528dee32 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:13.619910 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-6b592232-a678-42a9-9dc3-a7d4528dee32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:13.619910 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:13.619910 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:13.619910 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:13.619910 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:13.619910 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:13.619910 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:13.619910 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:13.619910 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:13.619910 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:13.619910 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:13.619910 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:13.619910 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:13.619910 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:13.619910 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:13.619910 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:13.621670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:13.621670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:13.621670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:13.621670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:13.621670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:13.621670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:13.621670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:13.621670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:13.621670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:13.621670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:13.621670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:13.621670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:13.621670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:13.621670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:13.621670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:13.621670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:13.621670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:13.622691 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:13.622691 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:13.622691 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:09:13.622691 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 616/1242] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:09:13.627997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d53d9bb-7cdb-47a9-9367-c03762a2dcf4 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:13.758750 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:13.765723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d53d9bb-7cdb-47a9-9367-c03762a2dcf4 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:13.807407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb3dff4-3c21-455a-8713-af58e79818fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:13.807978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3dff4-3c21-455a-8713-af58e79818fe None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:13.808316 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3dff4-3c21-455a-8713-af58e79818fe None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:13.809082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3dff4-3c21-455a-8713-af58e79818fe None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:13.809727 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb3dff4-3c21-455a-8713-af58e79818fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:13.810037 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb3dff4-3c21-455a-8713-af58e79818fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:13.811465 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb3dff4-3c21-455a-8713-af58e79818fe None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:13.825831 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 627/1243] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:09:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:13.832259 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b592232-a678-42a9-9dc3-a7d4528dee32 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:13.926235 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:13.957518 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b592232-a678-42a9-9dc3-a7d4528dee32 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:14.008904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df6571-e557-46b2-aeb4-9424acdfbe68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:14.009521 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df6571-e557-46b2-aeb4-9424acdfbe68 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:14.009836 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df6571-e557-46b2-aeb4-9424acdfbe68 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:14.010115 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df6571-e557-46b2-aeb4-9424acdfbe68 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:14.018921 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8df6571-e557-46b2-aeb4-9424acdfbe68 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:14.020668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8df6571-e557-46b2-aeb4-9424acdfbe68 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:14.035804 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-d8df6571-e557-46b2-aeb4-9424acdfbe68 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:09:14.035804 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:14.035804 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:14.035804 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:14.035804 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:14.035804 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:14.035804 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:14.035804 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:14.035804 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:14.035804 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:14.035804 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:14.035804 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:14.035804 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:09:14.035804 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:09:14.035804 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:09:14.036840 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:09:14.036840 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:14.036840 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:14.036840 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:14.036840 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:14.036840 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:14.036840 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:14.036840 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:14.036840 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:14.036840 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:14.036840 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:14.036840 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:14.036840 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:14.036840 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:14.036840 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:14.036840 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:14.036840 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:14.043911 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:09:14.043911 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:09:14.043911 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:09:14.043911 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:09:14.043911 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:09:14.043911 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:09:14.043911 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:09:14.043911 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:09:14.043911 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 617/1244] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:09:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:14.044689 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb3dff4-3c21-455a-8713-af58e79818fe None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:14.173935 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb3dff4-3c21-455a-8713-af58e79818fe None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:14.213536 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b1473b-0cdb-4f2a-a98b-44df8bfec1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:14.214147 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1473b-0cdb-4f2a-a98b-44df8bfec1e9 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:14.214421 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1473b-0cdb-4f2a-a98b-44df8bfec1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:14.215205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1473b-0cdb-4f2a-a98b-44df8bfec1e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:14.215933 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b1473b-0cdb-4f2a-a98b-44df8bfec1e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:14.216234 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b1473b-0cdb-4f2a-a98b-44df8bfec1e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:14.217679 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b1473b-0cdb-4f2a-a98b-44df8bfec1e9 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:14.231139 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 628/1245] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:09:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:14.237298 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df6571-e557-46b2-aeb4-9424acdfbe68 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:14.367764 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df6571-e557-46b2-aeb4-9424acdfbe68 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:14.410294 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7bb32-a829-4266-a899-11ee5a61909d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:14.411063 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7bb32-a829-4266-a899-11ee5a61909d None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:14.411380 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7bb32-a829-4266-a899-11ee5a61909d None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:14.411781 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7bb32-a829-4266-a899-11ee5a61909d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:14.412595 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca7bb32-a829-4266-a899-11ee5a61909d None admin] RBAC: Adding query filter params (role.id=8cf764a0144a490a822fcf86d7aceeb0, scope.project.id=fa3178684d184ec880f17640d19d55b7, user.id=524bcfffcdfa4790885e0d4cbd66fa80) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:14.412956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca7bb32-a829-4266-a899-11ee5a61909d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:14.414698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca7bb32-a829-4266-a899-11ee5a61909d None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:14.423573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 618/1246] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:09:14 2022] GET /identity/v3/role_assignments?user.id=524bcfffcdfa4790885e0d4cbd66fa80&scope.project.id=fa3178684d184ec880f17640d19d55b7&role.id=8cf764a0144a490a822fcf86d7aceeb0 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:16.599747 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c26090-7527-4154-a88a-5bb92204f2a1 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:16.600376 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c26090-7527-4154-a88a-5bb92204f2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:16.600376 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c26090-7527-4154-a88a-5bb92204f2a1 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:16.601702 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 629/1247] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:16.608467 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c7579-3caa-47ea-9ff3-b79d416048cf None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:16.609133 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c7579-3caa-47ea-9ff3-b79d416048cf None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:16.609296 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c7579-3caa-47ea-9ff3-b79d416048cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:16.617244 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:16.634373 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-ce1c7579-3caa-47ea-9ff3-b79d416048cf None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:16.635931 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-ce1c7579-3caa-47ea-9ff3-b79d416048cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:16.724119 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 619/1248] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:09:17.002913 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c26090-7527-4154-a88a-5bb92204f2a1 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:17.025283 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:17.139133 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c26090-7527-4154-a88a-5bb92204f2a1 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:17.187799 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34bf6a-e6f0-421b-ad5c-7efba387a770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:17.188494 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34bf6a-e6f0-421b-ad5c-7efba387a770 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:17.188803 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34bf6a-e6f0-421b-ad5c-7efba387a770 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:17.189149 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34bf6a-e6f0-421b-ad5c-7efba387a770 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:17.203715 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a34bf6a-e6f0-421b-ad5c-7efba387a770 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:17.208475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-9a34bf6a-e6f0-421b-ad5c-7efba387a770 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:09:17.208475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:17.208475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:17.208475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:17.208475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:17.208475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:17.208475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:17.208475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:17.208475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:17.208475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:17.208475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:17.208475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:17.208475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:17.208475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:17.208475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:17.208475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:17.209481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:17.209481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:17.209481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:17.209481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:17.209481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:17.209481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:17.209481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:17.209481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:17.209481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:17.209481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:17.209481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:17.209481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:17.209481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:17.209481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:17.209481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:17.209481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:17.210350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:17.210350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:17.210350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:17.210350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:17.210350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:17.210350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:17.210350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:17.210350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:17.210350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:17.210350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:17.210350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:09:17.210350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:09:17.210350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 630/1249] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:09:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:17.216003 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c7579-3caa-47ea-9ff3-b79d416048cf None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:17.338974 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c7579-3caa-47ea-9ff3-b79d416048cf None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:17.376759 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:17.383020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca219298-4f4d-41c3-985c-e3a31ef75506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:17.383702 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca219298-4f4d-41c3-985c-e3a31ef75506 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:17.383979 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca219298-4f4d-41c3-985c-e3a31ef75506 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:17.384369 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca219298-4f4d-41c3-985c-e3a31ef75506 None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:17.385279 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca219298-4f4d-41c3-985c-e3a31ef75506 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:17.385645 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca219298-4f4d-41c3-985c-e3a31ef75506 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:17.387620 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca219298-4f4d-41c3-985c-e3a31ef75506 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:17.396860 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 620/1250] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:17.403434 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34bf6a-e6f0-421b-ad5c-7efba387a770 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:17.498285 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:17.536053 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34bf6a-e6f0-421b-ad5c-7efba387a770 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:17.578841 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a54382-5191-4bc8-a721-4db8ec9774fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:17.579474 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a54382-5191-4bc8-a721-4db8ec9774fc None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:17.579786 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a54382-5191-4bc8-a721-4db8ec9774fc None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:17.580052 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a54382-5191-4bc8-a721-4db8ec9774fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:17.592458 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a54382-5191-4bc8-a721-4db8ec9774fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:17.593995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a54382-5191-4bc8-a721-4db8ec9774fc None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:17.612571 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-b5a54382-5191-4bc8-a721-4db8ec9774fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:17.612571 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:17.612571 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:17.612571 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:17.612571 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:17.612571 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:17.612571 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:17.612571 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:17.612571 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:17.612571 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:17.612571 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:17.612571 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:17.612571 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:17.612571 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:17.612571 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:17.612571 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:17.614108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:17.614108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:17.614108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:17.614108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:17.614108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:17.614108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:17.614108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:17.614108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:17.614108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:17.614108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:17.614108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:17.614108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:17.614108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:17.614108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:17.614108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:17.614108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:17.614108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:17.615044 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:17.615044 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:17.615044 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:09:17.615044 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 631/1251] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:17.621582 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca219298-4f4d-41c3-985c-e3a31ef75506 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:17.760832 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca219298-4f4d-41c3-985c-e3a31ef75506 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:17.802807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e993b46-b740-47c4-8756-c9ce63053ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:17.803572 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e993b46-b740-47c4-8756-c9ce63053ebd None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:17.803841 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e993b46-b740-47c4-8756-c9ce63053ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:17.804114 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e993b46-b740-47c4-8756-c9ce63053ebd None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:17.804773 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e993b46-b740-47c4-8756-c9ce63053ebd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:17.805076 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e993b46-b740-47c4-8756-c9ce63053ebd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:17.806569 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e993b46-b740-47c4-8756-c9ce63053ebd None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:17.823025 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 621/1252] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:17.829490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a54382-5191-4bc8-a721-4db8ec9774fc None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:17.952667 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a54382-5191-4bc8-a721-4db8ec9774fc None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:17.985772 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:17.995802 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3b178-92da-4343-bdf7-7113dd43a91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:17.996450 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3b178-92da-4343-bdf7-7113dd43a91b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:17.996729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3b178-92da-4343-bdf7-7113dd43a91b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:17.997055 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3b178-92da-4343-bdf7-7113dd43a91b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:18.005692 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b3b178-92da-4343-bdf7-7113dd43a91b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:18.007140 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b3b178-92da-4343-bdf7-7113dd43a91b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:18.019940 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-d1b3b178-92da-4343-bdf7-7113dd43a91b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 05 12:09:18.019940 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:18.019940 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:18.019940 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:18.019940 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:18.019940 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:18.019940 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:18.019940 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:18.019940 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:18.019940 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:18.019940 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:18.019940 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:18.019940 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:09:18.019940 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:09:18.019940 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:09:18.019940 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:09:18.027505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:18.027505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:18.027505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:09:18.027505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:09:18.027505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:09:18.027505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:09:18.027505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:18.027505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:18.027505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:18.027505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:18.027505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:18.027505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:18.027505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:18.027505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:18.027505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:18.027505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:18.027505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:18.027505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:18.028924 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:18.028924 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:18.028924 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:09:18.028924 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:09:18.028924 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:09:18.028924 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:09:18.028924 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:09:18.028924 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:09:18.028924 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 05 12:09:18.028924 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:09:18.028924 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 632/1253] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:18.029905 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e993b46-b740-47c4-8756-c9ce63053ebd None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:18.093943 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:18.155226 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e993b46-b740-47c4-8756-c9ce63053ebd None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:18.199667 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2072c-1dca-4cd3-803d-3312647800d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:18.200271 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2072c-1dca-4cd3-803d-3312647800d8 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:18.200551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2072c-1dca-4cd3-803d-3312647800d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:18.200816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2072c-1dca-4cd3-803d-3312647800d8 None admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:18.201554 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d2072c-1dca-4cd3-803d-3312647800d8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:18.201856 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d2072c-1dca-4cd3-803d-3312647800d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:18.203452 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d2072c-1dca-4cd3-803d-3312647800d8 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:18.224627 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 622/1254] 10.222.0.16 () {56 vars in 1143 bytes} [Mon Sep 5 12:09:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:18.231168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3b178-92da-4343-bdf7-7113dd43a91b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:18.361482 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3b178-92da-4343-bdf7-7113dd43a91b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:18.407084 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86fbcb-71b4-4c8d-858f-97856546c044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:18.407929 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86fbcb-71b4-4c8d-858f-97856546c044 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:18.408276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86fbcb-71b4-4c8d-858f-97856546c044 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:18.408521 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86fbcb-71b4-4c8d-858f-97856546c044 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:18.482850 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:18.499418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86fbcb-71b4-4c8d-858f-97856546c044 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:18.501193 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86fbcb-71b4-4c8d-858f-97856546c044 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:18.635803 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 633/1255] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:09:18.637978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2072c-1dca-4cd3-803d-3312647800d8 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:18.776010 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2072c-1dca-4cd3-803d-3312647800d8 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:18.795656 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:18.822958 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2680881-1120-4deb-a4e7-28c7a391dbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:18.823715 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2680881-1120-4deb-a4e7-28c7a391dbce None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:18.824651 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2680881-1120-4deb-a4e7-28c7a391dbce None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:18.824935 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2680881-1120-4deb-a4e7-28c7a391dbce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:18.837772 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2680881-1120-4deb-a4e7-28c7a391dbce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:18.839467 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2680881-1120-4deb-a4e7-28c7a391dbce None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:18.856426 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-f2680881-1120-4deb-a4e7-28c7a391dbce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:18.856426 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:18.856426 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:18.856426 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:18.856426 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:18.856426 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:18.856426 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:18.856426 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:18.856426 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:18.856426 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:18.856426 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:18.856426 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:18.856426 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:18.856426 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:18.856426 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:18.856426 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:18.857823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:18.857823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:18.857823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:18.857823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:18.857823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:18.857823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:18.857823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:18.857823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:18.857823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:18.857823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:18.857823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:18.857823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:18.857823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:18.857823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:18.857823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:18.857823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:18.857823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:18.858925 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:18.858925 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:18.858925 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:09:18.858925 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 623/1256] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:09:18.864842 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86fbcb-71b4-4c8d-858f-97856546c044 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:18.978665 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:18.994597 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86fbcb-71b4-4c8d-858f-97856546c044 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:19.039286 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96850bb8-9242-4ea2-9995-458c5ae9d5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:19.039944 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96850bb8-9242-4ea2-9995-458c5ae9d5b7 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:19.040261 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96850bb8-9242-4ea2-9995-458c5ae9d5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:19.040541 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96850bb8-9242-4ea2-9995-458c5ae9d5b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:19.041194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96850bb8-9242-4ea2-9995-458c5ae9d5b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:19.041498 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96850bb8-9242-4ea2-9995-458c5ae9d5b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:19.043397 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96850bb8-9242-4ea2-9995-458c5ae9d5b7 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:19.062228 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 634/1257] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:09:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:19.063622 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2680881-1120-4deb-a4e7-28c7a391dbce None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:19.201922 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2680881-1120-4deb-a4e7-28c7a391dbce None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:19.247189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2fd88-3124-46fb-84cf-1e92b3bbec17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:19.247884 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2fd88-3124-46fb-84cf-1e92b3bbec17 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:19.248171 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2fd88-3124-46fb-84cf-1e92b3bbec17 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:19.248442 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2fd88-3124-46fb-84cf-1e92b3bbec17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:19.257180 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa2fd88-3124-46fb-84cf-1e92b3bbec17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:19.258604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa2fd88-3124-46fb-84cf-1e92b3bbec17 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:19.271722 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-baa2fd88-3124-46fb-84cf-1e92b3bbec17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:09:19.271722 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:19.271722 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:19.271722 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:19.271722 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:19.271722 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:19.271722 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:19.271722 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:19.271722 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:19.271722 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:19.271722 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:19.271722 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:19.271722 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:09:19.271722 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:09:19.271722 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:09:19.272988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:09:19.272988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:19.272988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:19.272988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:19.272988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:19.272988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:19.272988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:19.272988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:19.272988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:19.272988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:19.272988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:19.272988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:19.272988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:19.272988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:19.272988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:19.272988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:19.272988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:19.274153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:09:19.274153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:09:19.274153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:09:19.274153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:09:19.274153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:09:19.274153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:09:19.274153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:09:19.274153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:09:19.274153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 624/1258] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:09:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:19.281490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96850bb8-9242-4ea2-9995-458c5ae9d5b7 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:19.429541 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96850bb8-9242-4ea2-9995-458c5ae9d5b7 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:19.480955 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70746ab0-7415-46ac-a527-d538cbebe462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:19.481729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70746ab0-7415-46ac-a527-d538cbebe462 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:19.482153 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70746ab0-7415-46ac-a527-d538cbebe462 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:19.482549 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70746ab0-7415-46ac-a527-d538cbebe462 None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:19.483489 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70746ab0-7415-46ac-a527-d538cbebe462 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:19.483945 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70746ab0-7415-46ac-a527-d538cbebe462 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:19.485660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70746ab0-7415-46ac-a527-d538cbebe462 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:19.504717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 635/1259] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:09:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:19.513979 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2fd88-3124-46fb-84cf-1e92b3bbec17 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:19.556138 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:19.653328 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2fd88-3124-46fb-84cf-1e92b3bbec17 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:19.699966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6002108b-35bc-4031-98be-e5e57bf0f2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:19.700858 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002108b-35bc-4031-98be-e5e57bf0f2df None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:19.701143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002108b-35bc-4031-98be-e5e57bf0f2df None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:19.701415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002108b-35bc-4031-98be-e5e57bf0f2df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:19.702157 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6002108b-35bc-4031-98be-e5e57bf0f2df None admin] RBAC: Adding query filter params (role.id=618c48706f7f445abeddf7eb2f8e927a, scope.project.id=fa3178684d184ec880f17640d19d55b7, user.id=524bcfffcdfa4790885e0d4cbd66fa80) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:19.702470 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6002108b-35bc-4031-98be-e5e57bf0f2df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:19.704345 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6002108b-35bc-4031-98be-e5e57bf0f2df None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:19.711261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 625/1260] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:09:19 2022] GET /identity/v3/role_assignments?user.id=524bcfffcdfa4790885e0d4cbd66fa80&scope.project.id=fa3178684d184ec880f17640d19d55b7&role.id=618c48706f7f445abeddf7eb2f8e927a => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:21.639617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a89584-f6eb-4469-aedc-90ff4bd0d8bb None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:21.640123 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a89584-f6eb-4469-aedc-90ff4bd0d8bb None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:21.640123 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a89584-f6eb-4469-aedc-90ff4bd0d8bb None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:21.641573 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 636/1261] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:21.647151 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b816f8-942a-472c-bc47-04c74a4d82bc None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:21.647925 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b816f8-942a-472c-bc47-04c74a4d82bc None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:21.647925 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b816f8-942a-472c-bc47-04c74a4d82bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:21.666725 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-20b816f8-942a-472c-bc47-04c74a4d82bc None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:21.668539 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-20b816f8-942a-472c-bc47-04c74a4d82bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:21.736773 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 626/1262] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:09:22.006347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a89584-f6eb-4469-aedc-90ff4bd0d8bb None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:22.017738 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:22.129326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a89584-f6eb-4469-aedc-90ff4bd0d8bb None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:22.175495 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace1390e-0ebc-49a0-8101-679cdad8ea21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:22.176444 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1390e-0ebc-49a0-8101-679cdad8ea21 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:22.176739 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1390e-0ebc-49a0-8101-679cdad8ea21 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:22.177022 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1390e-0ebc-49a0-8101-679cdad8ea21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:22.190485 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace1390e-0ebc-49a0-8101-679cdad8ea21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:22.195281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-ace1390e-0ebc-49a0-8101-679cdad8ea21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:09:22.195281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:22.195281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:22.195281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:22.195281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:22.195281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:22.195281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:22.195281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:22.195281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:22.195281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:22.195281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:22.195281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:22.195281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:22.195281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:22.195281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:22.195281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:22.196615 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:22.196615 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:22.196615 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:22.196615 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:22.196615 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:22.196615 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:22.196615 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:22.196615 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:22.196615 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:22.196615 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:22.196615 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:22.196615 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:22.196615 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:22.196615 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:22.196615 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:22.196615 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:22.197506 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:22.197506 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:22.197506 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:22.197506 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:22.197506 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:22.197506 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:22.197506 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:22.197506 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:22.197506 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:22.197506 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:22.197506 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:09:22.197506 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:09:22.197506 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 637/1263] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:09:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:09:22.202581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b816f8-942a-472c-bc47-04c74a4d82bc None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:22.221231 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:22.307128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b816f8-942a-472c-bc47-04c74a4d82bc None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:22.342424 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b4072-1ba2-4c92-a3fb-cfe13315eacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:22.343031 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b4072-1ba2-4c92-a3fb-cfe13315eacf None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:22.343316 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b4072-1ba2-4c92-a3fb-cfe13315eacf None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:22.343619 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b4072-1ba2-4c92-a3fb-cfe13315eacf None admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:22.344297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b4072-1ba2-4c92-a3fb-cfe13315eacf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:22.344601 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b4072-1ba2-4c92-a3fb-cfe13315eacf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:22.345978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b4072-1ba2-4c92-a3fb-cfe13315eacf None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:22.357103 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 627/1264] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:22.363078 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace1390e-0ebc-49a0-8101-679cdad8ea21 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:22.431062 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:22.469185 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace1390e-0ebc-49a0-8101-679cdad8ea21 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:22.508000 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f5c09-0d84-4e03-9abe-5a5f53065732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:22.508633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f5c09-0d84-4e03-9abe-5a5f53065732 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:22.508935 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f5c09-0d84-4e03-9abe-5a5f53065732 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:22.509203 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f5c09-0d84-4e03-9abe-5a5f53065732 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:22.519621 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1f5c09-0d84-4e03-9abe-5a5f53065732 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:22.521075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1f5c09-0d84-4e03-9abe-5a5f53065732 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:22.536163 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-2f1f5c09-0d84-4e03-9abe-5a5f53065732 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:22.536163 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:22.536163 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:22.536163 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:22.536163 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:22.536163 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:22.536163 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:22.536163 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:22.536163 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:22.536163 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:22.536163 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:22.536163 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:22.536163 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:22.536163 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:22.536163 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:22.536163 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:22.537384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:22.537384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:22.537384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:22.537384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:22.537384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:22.537384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:22.537384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:22.537384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:22.537384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:22.537384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:22.537384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:22.537384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:22.537384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:22.537384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:22.537384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:22.537384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:22.537384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:22.538319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:22.538319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:22.538319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:09:22.538319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 638/1265] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:22.543846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b4072-1ba2-4c92-a3fb-cfe13315eacf None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:22.632017 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:22.652601 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b4072-1ba2-4c92-a3fb-cfe13315eacf None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:22.686188 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b10b12-c280-417e-926a-17b5eb4e97e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:22.686797 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b10b12-c280-417e-926a-17b5eb4e97e9 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:22.687075 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b10b12-c280-417e-926a-17b5eb4e97e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:22.687348 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b10b12-c280-417e-926a-17b5eb4e97e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:22.688041 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b10b12-c280-417e-926a-17b5eb4e97e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:22.688348 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b10b12-c280-417e-926a-17b5eb4e97e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:22.689716 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b10b12-c280-417e-926a-17b5eb4e97e9 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:22.703205 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 628/1266] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:22.709626 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f5c09-0d84-4e03-9abe-5a5f53065732 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:22.814610 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f5c09-0d84-4e03-9abe-5a5f53065732 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:22.834705 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:22.853419 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492e09b-723a-4e6e-adbf-f0d0a94501a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:22.854065 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492e09b-723a-4e6e-adbf-f0d0a94501a6 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:22.854350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492e09b-723a-4e6e-adbf-f0d0a94501a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:22.854631 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492e09b-723a-4e6e-adbf-f0d0a94501a6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:22.863594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6492e09b-723a-4e6e-adbf-f0d0a94501a6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:22.865144 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6492e09b-723a-4e6e-adbf-f0d0a94501a6 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:22.877561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-6492e09b-723a-4e6e-adbf-f0d0a94501a6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 05 12:09:22.877561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:22.877561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:22.877561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:22.877561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:22.877561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:22.877561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:22.877561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:22.877561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:22.877561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:22.877561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:22.877561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:22.877561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:09:22.877561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:09:22.877561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:09:22.877561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:09:22.878836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:22.878836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:22.878836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:09:22.878836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:09:22.878836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:09:22.878836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:09:22.878836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:22.878836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:22.878836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:22.878836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:22.878836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:22.878836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:22.878836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:22.878836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:22.878836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:22.878836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:22.878836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:22.878836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:22.880172 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:22.880172 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:22.880172 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:09:22.880172 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:09:22.880172 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:09:22.880172 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:09:22.880172 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:09:22.880172 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:09:22.880172 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 05 12:09:22.880172 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:09:22.880172 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 639/1267] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:09:22.887605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b10b12-c280-417e-926a-17b5eb4e97e9 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:23.002320 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b10b12-c280-417e-926a-17b5eb4e97e9 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:23.039402 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df61a9b-dfa5-4151-96f1-1c8e83b34902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:23.040037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df61a9b-dfa5-4151-96f1-1c8e83b34902 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:23.040311 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df61a9b-dfa5-4151-96f1-1c8e83b34902 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:23.040582 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df61a9b-dfa5-4151-96f1-1c8e83b34902 None admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:23.041328 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df61a9b-dfa5-4151-96f1-1c8e83b34902 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:23.041638 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df61a9b-dfa5-4151-96f1-1c8e83b34902 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:23.043154 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df61a9b-dfa5-4151-96f1-1c8e83b34902 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:23.053136 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:23.065756 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 629/1268] 10.222.0.16 () {56 vars in 1143 bytes} [Mon Sep 5 12:09:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:23.071782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492e09b-723a-4e6e-adbf-f0d0a94501a6 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:23.195906 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492e09b-723a-4e6e-adbf-f0d0a94501a6 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:23.231293 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a014f4-4cfc-4200-af30-d110eea32396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:23.232025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a014f4-4cfc-4200-af30-d110eea32396 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:23.232345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a014f4-4cfc-4200-af30-d110eea32396 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:23.232660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a014f4-4cfc-4200-af30-d110eea32396 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:23.244265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a014f4-4cfc-4200-af30-d110eea32396 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:23.246236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a014f4-4cfc-4200-af30-d110eea32396 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:23.260984 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-87a014f4-4cfc-4200-af30-d110eea32396 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:23.260984 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:23.260984 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:23.260984 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:23.260984 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:23.260984 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:23.260984 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:23.260984 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:23.260984 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:23.260984 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:23.260984 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:23.260984 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:23.260984 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:23.260984 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:23.260984 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:23.260984 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:23.262782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:23.262782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:23.262782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:23.262782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:23.262782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:23.262782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:23.262782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:23.262782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:23.262782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:23.262782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:23.262782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:23.262782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:23.262782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:23.262782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:23.262782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:23.262782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:23.262782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:23.264156 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:23.264156 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:23.264156 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:09:23.264156 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 640/1269] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:23.268658 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df61a9b-dfa5-4151-96f1-1c8e83b34902 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:23.371270 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df61a9b-dfa5-4151-96f1-1c8e83b34902 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:23.407273 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537fb3f3-af2e-4c0d-89f9-90635d4b3597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:23.407904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537fb3f3-af2e-4c0d-89f9-90635d4b3597 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:23.408175 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537fb3f3-af2e-4c0d-89f9-90635d4b3597 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:23.408446 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537fb3f3-af2e-4c0d-89f9-90635d4b3597 None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:23.409076 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537fb3f3-af2e-4c0d-89f9-90635d4b3597 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:23.409369 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537fb3f3-af2e-4c0d-89f9-90635d4b3597 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:23.410710 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537fb3f3-af2e-4c0d-89f9-90635d4b3597 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:23.422467 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 630/1270] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:09:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:23.428625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a014f4-4cfc-4200-af30-d110eea32396 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:23.462533 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:23.533545 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a014f4-4cfc-4200-af30-d110eea32396 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:23.570423 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9000b20-942a-4e9b-abad-ee95554f629a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:23.571048 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9000b20-942a-4e9b-abad-ee95554f629a None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:23.571323 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9000b20-942a-4e9b-abad-ee95554f629a None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:23.571623 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9000b20-942a-4e9b-abad-ee95554f629a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:23.578446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9000b20-942a-4e9b-abad-ee95554f629a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:23.580086 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9000b20-942a-4e9b-abad-ee95554f629a None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:23.592006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-d9000b20-942a-4e9b-abad-ee95554f629a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:09:23.592006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:23.592006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:23.592006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:23.592006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:23.592006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:23.592006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:23.592006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:23.592006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:23.592006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:23.592006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:23.592006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:23.592006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:09:23.592006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:09:23.592006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:09:23.593251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:09:23.593251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:23.593251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:23.593251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:23.593251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:23.593251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:23.593251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:23.593251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:23.593251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:23.593251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:23.593251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:23.593251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:23.593251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:23.593251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:23.593251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:23.593251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:23.593251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:23.594376 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:09:23.594376 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:09:23.594376 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:09:23.594376 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:09:23.594376 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:09:23.594376 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:09:23.594376 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:09:23.594376 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:09:23.594376 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 641/1271] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:09:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:23.599197 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537fb3f3-af2e-4c0d-89f9-90635d4b3597 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:23.659654 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:23.705457 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537fb3f3-af2e-4c0d-89f9-90635d4b3597 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:23.749648 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f314536-c300-498e-87dd-5ea88092ca7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:23.750176 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f314536-c300-498e-87dd-5ea88092ca7c None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:23.750452 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f314536-c300-498e-87dd-5ea88092ca7c None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:23.750725 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f314536-c300-498e-87dd-5ea88092ca7c None admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:23.751443 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f314536-c300-498e-87dd-5ea88092ca7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:23.751777 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f314536-c300-498e-87dd-5ea88092ca7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:23.753215 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f314536-c300-498e-87dd-5ea88092ca7c None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:23.767139 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 631/1272] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:09:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:23.773564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9000b20-942a-4e9b-abad-ee95554f629a None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:23.876697 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:23.886614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9000b20-942a-4e9b-abad-ee95554f629a None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:23.923981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37a0500-0956-4845-9bf0-d41844059df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:23.924648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a0500-0956-4845-9bf0-d41844059df4 None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:23.924927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a0500-0956-4845-9bf0-d41844059df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:23.925200 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a0500-0956-4845-9bf0-d41844059df4 None admin] PATH_INFO: `/v3/projects/fa3178684d184ec880f17640d19d55b7/users/524bcfffcdfa4790885e0d4cbd66fa80/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:23.927194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37a0500-0956-4845-9bf0-d41844059df4 None admin] RBAC: Authorizing `identity:create_grant(project_id=fa3178684d184ec880f17640d19d55b7, user_id=524bcfffcdfa4790885e0d4cbd66fa80, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:23.928715 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37a0500-0956-4845-9bf0-d41844059df4 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:23.944065 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 642/1273] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Sep 5 12:09:23 2022] PUT /identity/v3/projects/fa3178684d184ec880f17640d19d55b7/users/524bcfffcdfa4790885e0d4cbd66fa80/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:09:25.894223 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de939ac-877d-4b4a-b98c-0aa1add4cae0 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:25.894791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de939ac-877d-4b4a-b98c-0aa1add4cae0 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:25.894791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de939ac-877d-4b4a-b98c-0aa1add4cae0 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:25.896148 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 632/1274] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:25.901558 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef7a3b-f567-4280-a94c-573ab67ed2c7 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:25.902045 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef7a3b-f567-4280-a94c-573ab67ed2c7 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:25.902167 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef7a3b-f567-4280-a94c-573ab67ed2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:25.921278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-68ef7a3b-f567-4280-a94c-573ab67ed2c7 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:25.922547 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-68ef7a3b-f567-4280-a94c-573ab67ed2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:25.992613 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 643/1275] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:09:26.282604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de939ac-877d-4b4a-b98c-0aa1add4cae0 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:26.393468 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de939ac-877d-4b4a-b98c-0aa1add4cae0 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:26.411637 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:26.437578 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c290c37-4344-4d1a-a876-59b7db421a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:26.438223 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c290c37-4344-4d1a-a876-59b7db421a39 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:26.438499 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c290c37-4344-4d1a-a876-59b7db421a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:26.438768 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c290c37-4344-4d1a-a876-59b7db421a39 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:26.449944 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c290c37-4344-4d1a-a876-59b7db421a39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:26.454872 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-5c290c37-4344-4d1a-a876-59b7db421a39 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:09:26.454872 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:26.454872 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:26.454872 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:26.454872 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:26.454872 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:26.454872 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:26.454872 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:26.454872 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:26.454872 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:26.454872 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:26.454872 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:26.454872 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:26.454872 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:26.454872 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:26.454872 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:26.456115 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:26.456115 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:26.456115 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:26.456115 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:26.456115 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:26.456115 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:26.456115 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:26.456115 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:26.456115 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:26.456115 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:26.456115 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:26.456115 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:26.456115 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:26.456115 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:26.456115 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:26.456115 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:26.456988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:26.456988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:26.456988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:26.456988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:26.456988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:26.456988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:26.456988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:26.456988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:26.456988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:26.456988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:26.456988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:09:26.456988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:09:26.456988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 633/1276] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Sep 5 12:09:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:26.462426 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef7a3b-f567-4280-a94c-573ab67ed2c7 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:26.546885 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:26.567793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef7a3b-f567-4280-a94c-573ab67ed2c7 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:26.603014 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfe59e-4ac7-4200-ac78-29c09eeb4404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:26.603658 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfe59e-4ac7-4200-ac78-29c09eeb4404 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:26.603944 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfe59e-4ac7-4200-ac78-29c09eeb4404 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:26.604214 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfe59e-4ac7-4200-ac78-29c09eeb4404 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:26.604886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bfe59e-4ac7-4200-ac78-29c09eeb4404 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:26.605695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bfe59e-4ac7-4200-ac78-29c09eeb4404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:26.607081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bfe59e-4ac7-4200-ac78-29c09eeb4404 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:26.616164 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 644/1277] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:26.622673 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c290c37-4344-4d1a-a876-59b7db421a39 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:26.762794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c290c37-4344-4d1a-a876-59b7db421a39 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:26.803744 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5bb53-2eeb-4b75-ae99-05d714694dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:26.804288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5bb53-2eeb-4b75-ae99-05d714694dca None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:26.804565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5bb53-2eeb-4b75-ae99-05d714694dca None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:26.804835 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5bb53-2eeb-4b75-ae99-05d714694dca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:26.815181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5bb53-2eeb-4b75-ae99-05d714694dca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:26.816662 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5bb53-2eeb-4b75-ae99-05d714694dca None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:26.832392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-dbc5bb53-2eeb-4b75-ae99-05d714694dca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:26.832392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:26.832392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:26.832392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:26.832392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:26.832392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:26.832392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:26.832392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:26.832392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:26.832392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:26.832392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:26.832392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:26.832392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:26.832392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:26.832392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:26.832392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:26.833687 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:26.833687 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:26.833687 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:26.833687 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:26.833687 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:26.833687 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:26.833687 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:26.833687 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:26.833687 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:26.833687 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:26.833687 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:26.833687 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:26.833687 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:26.833687 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:26.833687 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:26.833687 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:26.833687 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:26.834658 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:26.834658 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:26.834658 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:09:26.834658 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 634/1278] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:09:26.840212 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfe59e-4ac7-4200-ac78-29c09eeb4404 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:26.941111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfe59e-4ac7-4200-ac78-29c09eeb4404 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:26.974834 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d19b6-990c-4e87-bb99-5e3117b66a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:26.975435 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d19b6-990c-4e87-bb99-5e3117b66a11 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:26.975743 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d19b6-990c-4e87-bb99-5e3117b66a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:26.976015 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d19b6-990c-4e87-bb99-5e3117b66a11 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:26.976654 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d19b6-990c-4e87-bb99-5e3117b66a11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:26.976959 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d19b6-990c-4e87-bb99-5e3117b66a11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:26.978323 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d19b6-990c-4e87-bb99-5e3117b66a11 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:26.991333 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 645/1279] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:09:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:26.997959 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5bb53-2eeb-4b75-ae99-05d714694dca None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:27.031071 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:27.115214 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5bb53-2eeb-4b75-ae99-05d714694dca None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:27.148886 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4befa9-c9e8-440e-a4d1-2024daca6669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:27.149507 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4befa9-c9e8-440e-a4d1-2024daca6669 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:27.149783 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4befa9-c9e8-440e-a4d1-2024daca6669 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:27.150049 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4befa9-c9e8-440e-a4d1-2024daca6669 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:27.158480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4befa9-c9e8-440e-a4d1-2024daca6669 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:27.159929 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4befa9-c9e8-440e-a4d1-2024daca6669 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:27.170842 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-1c4befa9-c9e8-440e-a4d1-2024daca6669 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 05 12:09:27.170842 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:27.170842 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:27.170842 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:27.170842 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:27.170842 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:27.170842 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:27.170842 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:27.170842 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:27.170842 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:27.170842 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:27.170842 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:27.170842 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:09:27.170842 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:09:27.170842 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:09:27.170842 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:09:27.172040 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:27.172040 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:27.172040 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:09:27.172040 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:09:27.172040 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:09:27.172040 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:09:27.172040 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:27.172040 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:27.172040 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:27.172040 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:27.172040 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:27.172040 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:27.172040 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:27.172040 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:27.172040 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:27.172040 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:27.172040 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:27.172040 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:27.173019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:27.173019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:27.173019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:09:27.173019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:09:27.173019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:09:27.173019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:09:27.173019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:09:27.173019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:09:27.173019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 05 12:09:27.173019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:09:27.173019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 635/1280] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:27.177915 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d19b6-990c-4e87-bb99-5e3117b66a11 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:27.181707 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:27.278815 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d19b6-990c-4e87-bb99-5e3117b66a11 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:27.312329 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5bb717-776c-48ff-97ae-9ff3c1cf94fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:27.312924 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5bb717-776c-48ff-97ae-9ff3c1cf94fe None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:27.313207 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5bb717-776c-48ff-97ae-9ff3c1cf94fe None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:27.313477 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5bb717-776c-48ff-97ae-9ff3c1cf94fe None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:27.314197 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5bb717-776c-48ff-97ae-9ff3c1cf94fe None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:27.314501 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5bb717-776c-48ff-97ae-9ff3c1cf94fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:27.315952 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5bb717-776c-48ff-97ae-9ff3c1cf94fe None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:27.333558 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 646/1281] 10.222.0.16 () {56 vars in 1143 bytes} [Mon Sep 5 12:09:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:27.339335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4befa9-c9e8-440e-a4d1-2024daca6669 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:27.431741 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:27.442113 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4befa9-c9e8-440e-a4d1-2024daca6669 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:27.477612 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e1f6d-6c10-45d1-a7e3-0ae09f2ad586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:27.478213 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e1f6d-6c10-45d1-a7e3-0ae09f2ad586 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:27.478489 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e1f6d-6c10-45d1-a7e3-0ae09f2ad586 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:27.478763 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e1f6d-6c10-45d1-a7e3-0ae09f2ad586 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:27.548806 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e1f6d-6c10-45d1-a7e3-0ae09f2ad586 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:27.550247 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e1f6d-6c10-45d1-a7e3-0ae09f2ad586 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:27.649333 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:27.661081 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 636/1282] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:09:27.667308 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5bb717-776c-48ff-97ae-9ff3c1cf94fe None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:27.732957 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:27.774351 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5bb717-776c-48ff-97ae-9ff3c1cf94fe None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:27.813982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be452c64-7818-4555-af3d-04b3a0a47262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:27.814612 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be452c64-7818-4555-af3d-04b3a0a47262 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:27.814888 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be452c64-7818-4555-af3d-04b3a0a47262 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:27.815160 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be452c64-7818-4555-af3d-04b3a0a47262 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:27.826235 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be452c64-7818-4555-af3d-04b3a0a47262 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:27.827714 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be452c64-7818-4555-af3d-04b3a0a47262 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:27.842707 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-be452c64-7818-4555-af3d-04b3a0a47262 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:27.842707 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:27.842707 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:27.842707 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:27.842707 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:27.842707 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:27.842707 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:27.842707 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:27.842707 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:27.842707 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:27.842707 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:27.842707 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:27.842707 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:27.842707 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:27.842707 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:27.842707 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:27.844433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:27.844433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:27.844433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:27.844433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:27.844433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:27.844433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:27.844433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:27.844433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:27.844433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:27.844433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:27.844433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:27.844433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:27.844433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:27.844433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:27.844433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:27.844433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:27.844433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:27.846453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:27.846453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:27.846453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:09:27.846453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 647/1283] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:27.850976 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e1f6d-6c10-45d1-a7e3-0ae09f2ad586 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:27.956227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e1f6d-6c10-45d1-a7e3-0ae09f2ad586 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:27.991221 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d730ffd1-3e25-445a-af94-07987d88b06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:27.991849 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730ffd1-3e25-445a-af94-07987d88b06e None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:27.992125 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730ffd1-3e25-445a-af94-07987d88b06e None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:27.992394 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730ffd1-3e25-445a-af94-07987d88b06e None admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:27.993037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d730ffd1-3e25-445a-af94-07987d88b06e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:27.993335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d730ffd1-3e25-445a-af94-07987d88b06e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:27.994684 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d730ffd1-3e25-445a-af94-07987d88b06e None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:28.006820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 637/1284] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Sep 5 12:09:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:28.012566 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be452c64-7818-4555-af3d-04b3a0a47262 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:28.120080 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be452c64-7818-4555-af3d-04b3a0a47262 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:28.134408 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:28.157043 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4221f-f253-4776-b1bc-80ce610b9ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:28.157674 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4221f-f253-4776-b1bc-80ce610b9ae8 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:28.157947 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4221f-f253-4776-b1bc-80ce610b9ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:28.158212 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4221f-f253-4776-b1bc-80ce610b9ae8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:28.165047 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f4221f-f253-4776-b1bc-80ce610b9ae8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:28.166465 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f4221f-f253-4776-b1bc-80ce610b9ae8 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:28.179822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-f2f4221f-f253-4776-b1bc-80ce610b9ae8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:09:28.179822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:28.179822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:28.179822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:28.179822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:28.179822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:28.179822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:28.179822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:28.179822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:28.179822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:28.179822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:28.179822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:28.179822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:09:28.179822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:09:28.179822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:09:28.180985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:09:28.180985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:28.180985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:28.180985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:28.180985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:28.180985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:28.180985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:28.180985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:28.180985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:28.180985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:28.180985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:28.180985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:28.180985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:28.180985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:28.180985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:28.180985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:28.180985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:28.182213 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:09:28.182213 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:09:28.182213 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:09:28.182213 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:09:28.182213 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:09:28.182213 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:09:28.182213 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:09:28.182213 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 05 12:09:28.182213 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 648/1285] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Sep 5 12:09:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:28.187876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d730ffd1-3e25-445a-af94-07987d88b06e None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:28.239556 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:28.295471 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d730ffd1-3e25-445a-af94-07987d88b06e None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:28.330682 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b2070-358a-4b7b-a0e1-dd381d3ff9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:28.331273 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b2070-358a-4b7b-a0e1-dd381d3ff9c4 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:28.331605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b2070-358a-4b7b-a0e1-dd381d3ff9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:28.331878 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b2070-358a-4b7b-a0e1-dd381d3ff9c4 None admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:28.332575 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b2070-358a-4b7b-a0e1-dd381d3ff9c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:28.332875 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b2070-358a-4b7b-a0e1-dd381d3ff9c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:28.334295 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b2070-358a-4b7b-a0e1-dd381d3ff9c4 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:28.347013 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 638/1286] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Sep 5 12:09:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:28.353168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4221f-f253-4776-b1bc-80ce610b9ae8 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:28.455697 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4221f-f253-4776-b1bc-80ce610b9ae8 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:28.491883 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44428b62-7ca6-441e-8a3b-396d1c03e2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:28.492472 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44428b62-7ca6-441e-8a3b-396d1c03e2d2 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:28.492803 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44428b62-7ca6-441e-8a3b-396d1c03e2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:28.493126 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44428b62-7ca6-441e-8a3b-396d1c03e2d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:28.493907 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44428b62-7ca6-441e-8a3b-396d1c03e2d2 None admin] RBAC: Adding query filter params (role.id=618c48706f7f445abeddf7eb2f8e927a, scope.project.id=fa3178684d184ec880f17640d19d55b7, user.id=524bcfffcdfa4790885e0d4cbd66fa80) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:28.494263 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44428b62-7ca6-441e-8a3b-396d1c03e2d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:28.495848 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44428b62-7ca6-441e-8a3b-396d1c03e2d2 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:28.502602 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 649/1287] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Sep 5 12:09:28 2022] GET /identity/v3/role_assignments?user.id=524bcfffcdfa4790885e0d4cbd66fa80&scope.project.id=fa3178684d184ec880f17640d19d55b7&role.id=618c48706f7f445abeddf7eb2f8e927a => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:30.433075 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97f10d-dc88-4fbd-9c3d-988a0de6a4ac None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:30.433680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97f10d-dc88-4fbd-9c3d-988a0de6a4ac None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:30.433680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97f10d-dc88-4fbd-9c3d-988a0de6a4ac None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:30.435009 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 639/1288] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:09:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:30.440364 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c9cf3-8d3f-4c63-b435-9483bfd892d8 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:30.440778 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c9cf3-8d3f-4c63-b435-9483bfd892d8 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:30.440921 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c9cf3-8d3f-4c63-b435-9483bfd892d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:30.460112 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-409c9cf3-8d3f-4c63-b435-9483bfd892d8 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:30.461312 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-409c9cf3-8d3f-4c63-b435-9483bfd892d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:30.493658 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:30.538316 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 650/1289] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:09:30.821153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db97f10d-dc88-4fbd-9c3d-988a0de6a4ac None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:30.927354 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db97f10d-dc88-4fbd-9c3d-988a0de6a4ac None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:30.938560 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:30.963969 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c8908-3e16-41bd-b1c0-c857c17635d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:30.965294 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c8908-3e16-41bd-b1c0-c857c17635d6 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:30.965575 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c8908-3e16-41bd-b1c0-c857c17635d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:30.965847 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c8908-3e16-41bd-b1c0-c857c17635d6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:30.966449 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c8908-3e16-41bd-b1c0-c857c17635d6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:30.967884 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c8908-3e16-41bd-b1c0-c857c17635d6 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:30.977866 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-142c8908-3e16-41bd-b1c0-c857c17635d6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 12:09:30.977866 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:30.977866 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:09:30.977866 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:09:30.977866 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:09:30.977866 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:09:30.977866 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:09:30.977866 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:09:30.977866 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 12:09:30.977866 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:09:30.977866 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:09:30.977866 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:09:30.977866 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:30.977866 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:30.977866 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:30.977866 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:30.977866 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:30.979178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:30.979178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:30.979178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:30.979178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:30.979178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:30.979178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:30.979178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:09:30.979178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:09:30.979178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:09:30.979178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:09:30.979178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:30.979178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:30.979178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:30.979178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:30.979178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:30.979178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:30.980291 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:30.980291 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:30.980291 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:30.980291 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:30.980291 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:30.980291 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:30.980291 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:30.980291 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:30.980291 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:30.980291 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:30.980291 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:09:30.980291 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:09:30.980291 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 12:09:30.980291 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:09:30.983949 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 640/1290] 10.222.0.16 () {56 vars in 1117 bytes} [Mon Sep 5 12:09:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:30.985271 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c9cf3-8d3f-4c63-b435-9483bfd892d8 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:31.104706 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c9cf3-8d3f-4c63-b435-9483bfd892d8 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:31.153288 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f344d3-bd0c-43c4-b8e4-93aa898dc57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:31.153948 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f344d3-bd0c-43c4-b8e4-93aa898dc57f None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:31.154258 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f344d3-bd0c-43c4-b8e4-93aa898dc57f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:31.154849 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f344d3-bd0c-43c4-b8e4-93aa898dc57f None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:31.155567 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f344d3-bd0c-43c4-b8e4-93aa898dc57f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:31.155906 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f344d3-bd0c-43c4-b8e4-93aa898dc57f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:31.157420 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f344d3-bd0c-43c4-b8e4-93aa898dc57f None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:31.163466 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 651/1291] 10.222.0.16 () {56 vars in 1116 bytes} [Mon Sep 5 12:09:30 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:31.169601 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c8908-3e16-41bd-b1c0-c857c17635d6 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:31.279126 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c8908-3e16-41bd-b1c0-c857c17635d6 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:31.315252 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464a5e2-9af1-47e5-b05e-c2dd2b713fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:31.315950 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464a5e2-9af1-47e5-b05e-c2dd2b713fbb None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:31.316277 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464a5e2-9af1-47e5-b05e-c2dd2b713fbb None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:31.316612 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464a5e2-9af1-47e5-b05e-c2dd2b713fbb None admin] PATH_INFO: `/v3/services` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:31.317316 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464a5e2-9af1-47e5-b05e-c2dd2b713fbb None admin] RBAC: Adding query filter params (type=placement) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:31.317663 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464a5e2-9af1-47e5-b05e-c2dd2b713fbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:31.319099 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464a5e2-9af1-47e5-b05e-c2dd2b713fbb None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:31.324400 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 641/1292] 10.222.0.16 () {56 vars in 1116 bytes} [Mon Sep 5 12:09:31 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:33.307418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1d42e-dd1f-462d-ad34-b59f67fcd4a4 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:33.308049 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1d42e-dd1f-462d-ad34-b59f67fcd4a4 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:33.308049 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1d42e-dd1f-462d-ad34-b59f67fcd4a4 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:33.310004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 652/1293] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:09:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:09:33.315633 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb4e9e-7fa1-4209-b00b-a9941ecca730 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:33.316118 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb4e9e-7fa1-4209-b00b-a9941ecca730 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:33.316239 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb4e9e-7fa1-4209-b00b-a9941ecca730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:33.335618 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-03bb4e9e-7fa1-4209-b00b-a9941ecca730 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:33.336940 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-03bb4e9e-7fa1-4209-b00b-a9941ecca730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:33.376596 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:33.415810 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 642/1294] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:09:33.689744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1d42e-dd1f-462d-ad34-b59f67fcd4a4 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:33.723202 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:33.800602 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1d42e-dd1f-462d-ad34-b59f67fcd4a4 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:33.837997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23609bf9-067d-48a3-a79c-bf1d67b83527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:33.838611 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23609bf9-067d-48a3-a79c-bf1d67b83527 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:33.838893 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23609bf9-067d-48a3-a79c-bf1d67b83527 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:33.839169 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23609bf9-067d-48a3-a79c-bf1d67b83527 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:33.839927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23609bf9-067d-48a3-a79c-bf1d67b83527 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:33.841325 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23609bf9-067d-48a3-a79c-bf1d67b83527 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:33.858357 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 653/1295] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Sep 5 12:09:33 2022] POST /identity/v3/services => generated 246 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:09:35.814489 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33224482-ee21-4679-ac8f-2a92588bdc75 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:35.815033 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33224482-ee21-4679-ac8f-2a92588bdc75 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:35.815205 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33224482-ee21-4679-ac8f-2a92588bdc75 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:35.816602 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 643/1296] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:35.822384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7150b3c-0bd3-430f-a99d-4474f28628d7 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:35.822951 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7150b3c-0bd3-430f-a99d-4474f28628d7 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:35.822951 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7150b3c-0bd3-430f-a99d-4474f28628d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:35.842030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-c7150b3c-0bd3-430f-a99d-4474f28628d7 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:35.843240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c7150b3c-0bd3-430f-a99d-4474f28628d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:35.924529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 654/1297] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:35 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:09:36.203802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33224482-ee21-4679-ac8f-2a92588bdc75 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:36.310072 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33224482-ee21-4679-ac8f-2a92588bdc75 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:36.357702 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45bb3bd-a9e8-43fb-b8ac-5c3e2fde9903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:36.358679 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45bb3bd-a9e8-43fb-b8ac-5c3e2fde9903 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:36.359074 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45bb3bd-a9e8-43fb-b8ac-5c3e2fde9903 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:36.359358 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45bb3bd-a9e8-43fb-b8ac-5c3e2fde9903 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:36.360238 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45bb3bd-a9e8-43fb-b8ac-5c3e2fde9903 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:36.362434 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45bb3bd-a9e8-43fb-b8ac-5c3e2fde9903 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:36.375455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-d45bb3bd-a9e8-43fb-b8ac-5c3e2fde9903 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 12:09:36.375455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:36.375455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:09:36.375455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:09:36.375455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:09:36.375455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:09:36.375455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:09:36.375455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:09:36.375455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 12:09:36.375455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:09:36.375455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:09:36.375455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:09:36.375455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:36.375455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:36.375455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:36.375455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:36.375455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:36.376726 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:36.376726 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:36.376726 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:36.376726 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:36.376726 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:36.376726 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:36.376726 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:09:36.376726 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:09:36.376726 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:09:36.376726 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:09:36.376726 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:36.376726 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:36.376726 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:36.376726 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:36.376726 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:36.376726 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:36.377680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:36.377680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:36.377680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:36.377680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:36.377680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:36.377680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:36.377680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:36.377680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:36.377680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:36.377680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:36.377680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:09:36.377680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:09:36.377680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 12:09:36.377680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:09:36.377680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 644/1298] 10.222.0.16 () {56 vars in 1117 bytes} [Mon Sep 5 12:09:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:09:36.383239 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7150b3c-0bd3-430f-a99d-4474f28628d7 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:36.407222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:36.547332 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7150b3c-0bd3-430f-a99d-4474f28628d7 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:36.607321 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71828cd-258e-41e0-8bb3-b251141cb498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:36.607958 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71828cd-258e-41e0-8bb3-b251141cb498 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:36.608241 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71828cd-258e-41e0-8bb3-b251141cb498 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:36.608515 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71828cd-258e-41e0-8bb3-b251141cb498 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:36.609150 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71828cd-258e-41e0-8bb3-b251141cb498 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:36.609456 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71828cd-258e-41e0-8bb3-b251141cb498 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:36.610863 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71828cd-258e-41e0-8bb3-b251141cb498 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:36.616533 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 655/1299] 10.222.0.16 () {56 vars in 1116 bytes} [Mon Sep 5 12:09:36 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:36.623584 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45bb3bd-a9e8-43fb-b8ac-5c3e2fde9903 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:36.656345 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:36.725380 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45bb3bd-a9e8-43fb-b8ac-5c3e2fde9903 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:36.758740 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c4b7a5-4fa9-470f-85e9-61f76889f0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:36.759336 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4b7a5-4fa9-470f-85e9-61f76889f0cb None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:36.759642 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4b7a5-4fa9-470f-85e9-61f76889f0cb None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:36.759910 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4b7a5-4fa9-470f-85e9-61f76889f0cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:36.760611 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c4b7a5-4fa9-470f-85e9-61f76889f0cb None admin] RBAC: Adding query filter params (interface=public, service_id=7da7f13f5b0d4bed8519bd29bf21c3e4, region_id=RegionOne) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:36.760914 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c4b7a5-4fa9-470f-85e9-61f76889f0cb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:36.762986 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c4b7a5-4fa9-470f-85e9-61f76889f0cb None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:36.768531 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 645/1300] 10.222.0.16 () {56 vars in 1251 bytes} [Mon Sep 5 12:09:36 2022] GET /identity/v3/endpoints?service_id=7da7f13f5b0d4bed8519bd29bf21c3e4&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:36.777589 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71828cd-258e-41e0-8bb3-b251141cb498 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:36.862507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:36.881735 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71828cd-258e-41e0-8bb3-b251141cb498 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:36.915555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca208303-f381-401e-9370-7dc408fa0387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:36.916157 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca208303-f381-401e-9370-7dc408fa0387 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:36.916432 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca208303-f381-401e-9370-7dc408fa0387 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:36.916699 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca208303-f381-401e-9370-7dc408fa0387 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:36.917299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca208303-f381-401e-9370-7dc408fa0387 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:36.919210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca208303-f381-401e-9370-7dc408fa0387 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:36.926364 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 656/1301] 10.222.0.16 () {56 vars in 1087 bytes} [Mon Sep 5 12:09:36 2022] GET /identity/v3/services => generated 2191 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:09:38.837767 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edad84-7622-46b0-be41-768647beb838 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:38.838306 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edad84-7622-46b0-be41-768647beb838 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:38.838306 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edad84-7622-46b0-be41-768647beb838 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:38.840176 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 646/1302] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:09:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:09:38.846383 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa01933-9d1b-4ef0-acdd-5c4a576f4104 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:38.846857 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa01933-9d1b-4ef0-acdd-5c4a576f4104 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:38.846955 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa01933-9d1b-4ef0-acdd-5c4a576f4104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:38.865389 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-aaa01933-9d1b-4ef0-acdd-5c4a576f4104 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:38.866558 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-aaa01933-9d1b-4ef0-acdd-5c4a576f4104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:38.940911 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 657/1303] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:38 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:09:39.208704 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8edad84-7622-46b0-be41-768647beb838 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:39.308139 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:39.315442 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8edad84-7622-46b0-be41-768647beb838 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:39.349491 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1266ba50-be31-4259-86dd-fb2d90d1de27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:39.350107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266ba50-be31-4259-86dd-fb2d90d1de27 None admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:39.350379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266ba50-be31-4259-86dd-fb2d90d1de27 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:39.350644 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266ba50-be31-4259-86dd-fb2d90d1de27 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:39.351223 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1266ba50-be31-4259-86dd-fb2d90d1de27 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:39.352601 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1266ba50-be31-4259-86dd-fb2d90d1de27 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:39.361413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application [None req-1266ba50-be31-4259-86dd-fb2d90d1de27 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 12:09:39.361413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:39.361413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:09:39.361413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:09:39.361413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:09:39.361413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:09:39.361413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:09:39.361413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:09:39.361413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 12:09:39.361413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:09:39.361413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:09:39.361413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:09:39.361413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:39.361413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:39.361413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:39.361413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:39.361413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:39.362765 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:39.362765 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:39.362765 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:39.362765 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:39.362765 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:39.362765 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:39.362765 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:09:39.362765 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:09:39.362765 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:09:39.362765 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:09:39.362765 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:39.362765 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:39.362765 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:39.362765 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:39.362765 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:39.362765 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:39.363903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:39.363903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:39.363903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:39.363903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:39.363903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:39.363903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:39.363903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:39.363903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:39.363903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:39.363903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:39.363903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:09:39.363903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:09:39.363903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 12:09:39.363903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: ERROR keystone.server.flask.application Sep 05 12:09:39.363903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 647/1304] 10.222.0.16 () {56 vars in 1117 bytes} [Mon Sep 5 12:09:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:09:39.368795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa01933-9d1b-4ef0-acdd-5c4a576f4104 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:39.446600 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:39.471676 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa01933-9d1b-4ef0-acdd-5c4a576f4104 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:39.505079 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87555cba-a931-4712-9548-1ee92b05efaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:39.505695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87555cba-a931-4712-9548-1ee92b05efaf None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:39.505977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87555cba-a931-4712-9548-1ee92b05efaf None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:39.506253 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87555cba-a931-4712-9548-1ee92b05efaf None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:39.506880 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87555cba-a931-4712-9548-1ee92b05efaf None admin] RBAC: Adding query filter params (name=placement) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:39.507183 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87555cba-a931-4712-9548-1ee92b05efaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:39.508587 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87555cba-a931-4712-9548-1ee92b05efaf None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:39.514197 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 658/1305] 10.222.0.16 () {56 vars in 1116 bytes} [Mon Sep 5 12:09:39 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:39.520251 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1266ba50-be31-4259-86dd-fb2d90d1de27 None admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:39.625513 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1266ba50-be31-4259-86dd-fb2d90d1de27 None admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:39.660013 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce8877-2f61-4454-80ce-00efe73e19b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:39.660615 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce8877-2f61-4454-80ce-00efe73e19b8 None admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:39.660885 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce8877-2f61-4454-80ce-00efe73e19b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:39.661153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce8877-2f61-4454-80ce-00efe73e19b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:39.661819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce8877-2f61-4454-80ce-00efe73e19b8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:39.663183 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce8877-2f61-4454-80ce-00efe73e19b8 None admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:39.682520 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 648/1306] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:09:39 2022] POST /identity/v3/endpoints => generated 331 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:09:49.510807 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=30 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:49.679060 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:47.553476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030368bd-5dc3-4646-b753-2730bdf7244f None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:47.554404 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030368bd-5dc3-4646-b753-2730bdf7244f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:47.554844 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030368bd-5dc3-4646-b753-2730bdf7244f None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:47.560792 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 659/1307] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:10:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:47.562542 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802b9dd-65ab-4998-9eb1-e3dc01814bdb None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:47.563340 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802b9dd-65ab-4998-9eb1-e3dc01814bdb None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:47.563807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802b9dd-65ab-4998-9eb1-e3dc01814bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:47.654652 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-c802b9dd-65ab-4998-9eb1-e3dc01814bdb None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:47.656503 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-c802b9dd-65ab-4998-9eb1-e3dc01814bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:47.954177 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 649/1308] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:10:48.246193 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef7add-b825-4d00-a14f-034404e4e963 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:48.247296 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef7add-b825-4d00-a14f-034404e4e963 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:48.247870 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef7add-b825-4d00-a14f-034404e4e963 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:48.249820 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 660/1309] 10.222.0.16 () {54 vars in 965 bytes} [Mon Sep 5 12:10:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:48.258933 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f33db-7de7-4cb7-8e25-f25079985715 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:48.259409 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f33db-7de7-4cb7-8e25-f25079985715 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:48.260145 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f33db-7de7-4cb7-8e25-f25079985715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:48.381514 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-fa0f33db-7de7-4cb7-8e25-f25079985715 None None] MFA Rules not processed for user `9f79599ed0b74bdbab198627ef72252a`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:48.383347 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-fa0f33db-7de7-4cb7-8e25-f25079985715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:48.419340 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:48.489546 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 650/1310] 10.222.0.16 () {58 vars in 1062 bytes} [Mon Sep 5 12:10:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:10:48.498455 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef7add-b825-4d00-a14f-034404e4e963 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:48.500303 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ddef7add-b825-4d00-a14f-034404e4e963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:48.614079 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef7add-b825-4d00-a14f-034404e4e963 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:48.693803 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db06d4-5e46-4c6d-a10c-b60ff12ac8b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:48.694429 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db06d4-5e46-4c6d-a10c-b60ff12ac8b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:48.694777 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db06d4-5e46-4c6d-a10c-b60ff12ac8b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:48.695109 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db06d4-5e46-4c6d-a10c-b60ff12ac8b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:48.696836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e9db06d4-5e46-4c6d-a10c-b60ff12ac8b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:48.755392 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:48.815900 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db06d4-5e46-4c6d-a10c-b60ff12ac8b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:48.817425 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db06d4-5e46-4c6d-a10c-b60ff12ac8b6 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:48.838122 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 661/1311] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:10:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:10:51.086164 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bdaa59-acf6-4335-8ae0-789a3709c0fa None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:51.086634 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bdaa59-acf6-4335-8ae0-789a3709c0fa None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:51.086741 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bdaa59-acf6-4335-8ae0-789a3709c0fa None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:51.088086 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 651/1312] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:10:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:51.093998 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ccc24a-c15d-4d5a-a755-3da983bd4e9b None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:51.094844 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ccc24a-c15d-4d5a-a755-3da983bd4e9b None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:51.094844 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ccc24a-c15d-4d5a-a755-3da983bd4e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:51.182903 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-33ccc24a-c15d-4d5a-a755-3da983bd4e9b None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:51.184150 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-33ccc24a-c15d-4d5a-a755-3da983bd4e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:51.282210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 662/1313] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:10:51.564697 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bdaa59-acf6-4335-8ae0-789a3709c0fa None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:51.566306 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-b0bdaa59-acf6-4335-8ae0-789a3709c0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:51.670158 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bdaa59-acf6-4335-8ae0-789a3709c0fa None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:51.741681 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:51.744473 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa256c95-567a-400d-8cea-2ff1743fa160 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:51.745116 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa256c95-567a-400d-8cea-2ff1743fa160 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:51.745467 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa256c95-567a-400d-8cea-2ff1743fa160 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:51.746429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa256c95-567a-400d-8cea-2ff1743fa160 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:51.748551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-aa256c95-567a-400d-8cea-2ff1743fa160 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:51.858408 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa256c95-567a-400d-8cea-2ff1743fa160 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:51.860320 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa256c95-567a-400d-8cea-2ff1743fa160 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:51.880225 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 652/1314] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:10:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:10:54.127811 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15f2f3-94b1-4062-8fa4-418f60ce1a5e None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:54.128398 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15f2f3-94b1-4062-8fa4-418f60ce1a5e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:54.128398 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15f2f3-94b1-4062-8fa4-418f60ce1a5e None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:54.129755 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 663/1315] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:10:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:54.135840 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959574c-3440-4f89-a63c-bc7df810add3 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:54.136916 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959574c-3440-4f89-a63c-bc7df810add3 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:54.136916 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959574c-3440-4f89-a63c-bc7df810add3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:54.213479 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-6959574c-3440-4f89-a63c-bc7df810add3 None None] MFA Rules not processed for user `10881ece7d184527a88b151aac434c0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:54.215338 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-6959574c-3440-4f89-a63c-bc7df810add3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:54.273296 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:54.328594 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 653/1316] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:54 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:10:54.616058 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15f2f3-94b1-4062-8fa4-418f60ce1a5e None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:54.617698 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9c15f2f3-94b1-4062-8fa4-418f60ce1a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:54.628039 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:54.722136 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15f2f3-94b1-4062-8fa4-418f60ce1a5e None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:54.793692 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869f19e-eb8d-4f20-916d-c0f4db091860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:54.794293 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869f19e-eb8d-4f20-916d-c0f4db091860 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:54.794567 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869f19e-eb8d-4f20-916d-c0f4db091860 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:54.794836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869f19e-eb8d-4f20-916d-c0f4db091860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:54.796466 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7869f19e-eb8d-4f20-916d-c0f4db091860 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:54.856535 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:54.902747 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7869f19e-eb8d-4f20-916d-c0f4db091860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:54.904130 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7869f19e-eb8d-4f20-916d-c0f4db091860 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:54.922950 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 664/1317] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:10:54 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:10:57.430895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce9361-29ca-4480-8178-cad6ffa44f53 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:57.431428 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce9361-29ca-4480-8178-cad6ffa44f53 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:57.431428 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce9361-29ca-4480-8178-cad6ffa44f53 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:57.432806 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 654/1318] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:10:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:57.438117 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4c298-4043-4a26-a4a7-29b15b115ba5 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:57.438593 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4c298-4043-4a26-a4a7-29b15b115ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:57.438715 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4c298-4043-4a26-a4a7-29b15b115ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:57.508302 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-85e4c298-4043-4a26-a4a7-29b15b115ba5 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:57.509527 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-85e4c298-4043-4a26-a4a7-29b15b115ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:57.615541 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 665/1319] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:10:57.896760 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce9361-29ca-4480-8178-cad6ffa44f53 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:57.898291 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-e7ce9361-29ca-4480-8178-cad6ffa44f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:58.002095 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce9361-29ca-4480-8178-cad6ffa44f53 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:58.077632 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b6a40-bc04-489d-8074-1ff6b893479a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:58.078627 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b6a40-bc04-489d-8074-1ff6b893479a service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:58.079365 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b6a40-bc04-489d-8074-1ff6b893479a service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:58.080125 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b6a40-bc04-489d-8074-1ff6b893479a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:58.082439 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-a82b6a40-bc04-489d-8074-1ff6b893479a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:58.090434 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:58.205990 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b6a40-bc04-489d-8074-1ff6b893479a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:58.207342 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b6a40-bc04-489d-8074-1ff6b893479a service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:58.225408 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 655/1320] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:10:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:11:00.775197 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc5ff9-756d-4080-8ef9-4004e706f96f None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:00.775874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc5ff9-756d-4080-8ef9-4004e706f96f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:00.775874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc5ff9-756d-4080-8ef9-4004e706f96f None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:00.777569 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 666/1321] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:11:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:00.783016 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223a6ed-3aa2-4da1-b945-36d51b30f122 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:00.783552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223a6ed-3aa2-4da1-b945-36d51b30f122 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:00.783659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223a6ed-3aa2-4da1-b945-36d51b30f122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:00.851413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-7223a6ed-3aa2-4da1-b945-36d51b30f122 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:00.852662 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-7223a6ed-3aa2-4da1-b945-36d51b30f122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:00.911918 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:00.953284 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 656/1322] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:11:01.238865 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc5ff9-756d-4080-8ef9-4004e706f96f None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:01.240469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-cffc5ff9-756d-4080-8ef9-4004e706f96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:01.262263 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:01.354535 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc5ff9-756d-4080-8ef9-4004e706f96f None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:01.438215 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58973d5-6f12-4275-9565-81a6b8826247 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:01.438847 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58973d5-6f12-4275-9565-81a6b8826247 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:01.439127 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58973d5-6f12-4275-9565-81a6b8826247 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:01.439405 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58973d5-6f12-4275-9565-81a6b8826247 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:01.441077 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c58973d5-6f12-4275-9565-81a6b8826247 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:01.514275 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:01.553991 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58973d5-6f12-4275-9565-81a6b8826247 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:01.555421 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58973d5-6f12-4275-9565-81a6b8826247 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:01.575330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 667/1323] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:11:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:11:03.575041 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9699d9-8b2b-47e2-ba5d-a50cff031a84 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:03.575594 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9699d9-8b2b-47e2-ba5d-a50cff031a84 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:03.575594 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9699d9-8b2b-47e2-ba5d-a50cff031a84 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:03.577027 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 657/1324] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:11:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:03.582527 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce91c7-9360-4889-bce0-c6feceaabdae None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:03.582926 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce91c7-9360-4889-bce0-c6feceaabdae None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:03.583200 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce91c7-9360-4889-bce0-c6feceaabdae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:03.649596 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-31ce91c7-9360-4889-bce0-c6feceaabdae None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:03.650684 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-31ce91c7-9360-4889-bce0-c6feceaabdae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:03.750138 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 668/1325] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:11:04.046895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48c40c-d4bf-45a2-abaf-97614f580e92 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:04.047178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48c40c-d4bf-45a2-abaf-97614f580e92 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:04.047446 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48c40c-d4bf-45a2-abaf-97614f580e92 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:04.048883 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 658/1326] 10.222.0.16 () {54 vars in 965 bytes} [Mon Sep 5 12:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:04.055837 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8fa86-8296-4526-8059-b147155bac0b None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:04.055837 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8fa86-8296-4526-8059-b147155bac0b None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:04.056294 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8fa86-8296-4526-8059-b147155bac0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:04.092226 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:04.140407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-92f8fa86-8296-4526-8059-b147155bac0b None None] MFA Rules not processed for user `9f79599ed0b74bdbab198627ef72252a`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:04.141590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-92f8fa86-8296-4526-8059-b147155bac0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:04.229111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 669/1327] 10.222.0.16 () {58 vars in 1062 bytes} [Mon Sep 5 12:11:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:11:04.238878 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da48c40c-d4bf-45a2-abaf-97614f580e92 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:04.240496 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-da48c40c-d4bf-45a2-abaf-97614f580e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:04.362221 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da48c40c-d4bf-45a2-abaf-97614f580e92 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:04.444806 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98eb69f-bda8-47b8-8c6d-8c11c9083297 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:04.445412 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98eb69f-bda8-47b8-8c6d-8c11c9083297 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:04.445688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98eb69f-bda8-47b8-8c6d-8c11c9083297 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:04.445957 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98eb69f-bda8-47b8-8c6d-8c11c9083297 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:04.447595 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-b98eb69f-bda8-47b8-8c6d-8c11c9083297 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:04.465873 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:04.561021 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98eb69f-bda8-47b8-8c6d-8c11c9083297 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:04.562369 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98eb69f-bda8-47b8-8c6d-8c11c9083297 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:04.580431 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 659/1328] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:11:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:11:06.804631 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7f7fc-2071-43b1-98f1-fead6a10f86d None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:06.805162 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7f7fc-2071-43b1-98f1-fead6a10f86d None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:06.805162 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7f7fc-2071-43b1-98f1-fead6a10f86d None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:06.806524 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 670/1329] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:11:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:06.811870 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b391589-31c5-4e25-904e-ce809c3eefb3 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:06.812314 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b391589-31c5-4e25-904e-ce809c3eefb3 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:06.812430 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b391589-31c5-4e25-904e-ce809c3eefb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:06.878267 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-2b391589-31c5-4e25-904e-ce809c3eefb3 None None] MFA Rules not processed for user `10881ece7d184527a88b151aac434c0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:06.879472 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-2b391589-31c5-4e25-904e-ce809c3eefb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:06.966453 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:07.003726 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 660/1330] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:06 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:11:07.293870 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7f7fc-2071-43b1-98f1-fead6a10f86d None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:07.295422 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e9c7f7fc-2071-43b1-98f1-fead6a10f86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:07.423399 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7f7fc-2071-43b1-98f1-fead6a10f86d None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:07.446537 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:07.501125 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e1efc-c66b-49d6-8d13-fa54df1eb71e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:07.501724 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e1efc-c66b-49d6-8d13-fa54df1eb71e service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:07.501997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e1efc-c66b-49d6-8d13-fa54df1eb71e service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:07.502264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e1efc-c66b-49d6-8d13-fa54df1eb71e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:07.503889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8b0e1efc-c66b-49d6-8d13-fa54df1eb71e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:07.629748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e1efc-c66b-49d6-8d13-fa54df1eb71e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:07.631196 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e1efc-c66b-49d6-8d13-fa54df1eb71e service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:07.652028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 671/1331] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:11:07 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:11:09.983255 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09aca5b-c416-4fcf-8e0e-52028cd49751 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:09.983816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09aca5b-c416-4fcf-8e0e-52028cd49751 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:09.983816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09aca5b-c416-4fcf-8e0e-52028cd49751 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:09.985211 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 661/1332] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:11:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:09.990562 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f972be2-6dde-41c4-a7ef-65826da79c8a None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:09.990986 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f972be2-6dde-41c4-a7ef-65826da79c8a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:09.991109 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f972be2-6dde-41c4-a7ef-65826da79c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:10.054796 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:10.057943 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-3f972be2-6dde-41c4-a7ef-65826da79c8a None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:10.060414 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3f972be2-6dde-41c4-a7ef-65826da79c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:10.156475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 672/1333] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:11:10.437947 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09aca5b-c416-4fcf-8e0e-52028cd49751 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:10.439480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-d09aca5b-c416-4fcf-8e0e-52028cd49751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:10.544927 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09aca5b-c416-4fcf-8e0e-52028cd49751 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:10.616026 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac741af5-82d2-444b-86cc-af427922a2ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:10.616565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac741af5-82d2-444b-86cc-af427922a2ad service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:10.616836 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac741af5-82d2-444b-86cc-af427922a2ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:10.617104 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac741af5-82d2-444b-86cc-af427922a2ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:10.618685 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-ac741af5-82d2-444b-86cc-af427922a2ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:10.646117 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:10.728151 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac741af5-82d2-444b-86cc-af427922a2ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:10.729589 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac741af5-82d2-444b-86cc-af427922a2ad service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:10.747180 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 662/1334] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:11:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:11:12.711001 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931de0d-d4b4-465f-bb8a-abd0c08d82ef None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:12.711647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931de0d-d4b4-465f-bb8a-abd0c08d82ef None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:12.711647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931de0d-d4b4-465f-bb8a-abd0c08d82ef None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:12.712872 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 673/1335] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:12.718845 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae51f94e-2ad2-4afc-a13e-35732481df49 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:12.719335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae51f94e-2ad2-4afc-a13e-35732481df49 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:12.719441 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae51f94e-2ad2-4afc-a13e-35732481df49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:12.796901 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-ae51f94e-2ad2-4afc-a13e-35732481df49 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:12.798129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-ae51f94e-2ad2-4afc-a13e-35732481df49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:12.872689 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:12.896251 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 663/1336] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:11:13.186213 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931de0d-d4b4-465f-bb8a-abd0c08d82ef None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:13.187755 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c931de0d-d4b4-465f-bb8a-abd0c08d82ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:13.295878 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931de0d-d4b4-465f-bb8a-abd0c08d82ef None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:13.331607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:13.377048 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a0f68-eb66-4a6b-9728-61fe5dd3e1fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:13.377660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a0f68-eb66-4a6b-9728-61fe5dd3e1fb service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:13.377939 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a0f68-eb66-4a6b-9728-61fe5dd3e1fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:13.378214 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a0f68-eb66-4a6b-9728-61fe5dd3e1fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:13.379858 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-410a0f68-eb66-4a6b-9728-61fe5dd3e1fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:13.487705 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410a0f68-eb66-4a6b-9728-61fe5dd3e1fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:13.489138 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410a0f68-eb66-4a6b-9728-61fe5dd3e1fb service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:13.507815 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 674/1337] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:11:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:11:15.934758 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe3a69-b468-4676-a5ca-8b3b75bbf50f None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:15.935348 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe3a69-b468-4676-a5ca-8b3b75bbf50f None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:15.935758 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe3a69-b468-4676-a5ca-8b3b75bbf50f None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:15.938287 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 664/1338] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:11:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:15.944174 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547dc6a-4d06-4fa5-a2de-a049ac077801 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:15.944781 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547dc6a-4d06-4fa5-a2de-a049ac077801 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:15.944781 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547dc6a-4d06-4fa5-a2de-a049ac077801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:16.014298 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-8547dc6a-4d06-4fa5-a2de-a049ac077801 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:16.015493 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8547dc6a-4d06-4fa5-a2de-a049ac077801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:16.023836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:16.115517 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 675/1339] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:11:16.397148 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe3a69-b468-4676-a5ca-8b3b75bbf50f None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:16.398673 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-88fe3a69-b468-4676-a5ca-8b3b75bbf50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:16.516576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe3a69-b468-4676-a5ca-8b3b75bbf50f None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:16.593074 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6315ed7-17c8-4bb0-9fcb-73aabe6c596d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:16.593681 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6315ed7-17c8-4bb0-9fcb-73aabe6c596d service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:16.593959 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6315ed7-17c8-4bb0-9fcb-73aabe6c596d service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:16.594230 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6315ed7-17c8-4bb0-9fcb-73aabe6c596d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:16.595885 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-d6315ed7-17c8-4bb0-9fcb-73aabe6c596d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:16.635152 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:16.710267 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6315ed7-17c8-4bb0-9fcb-73aabe6c596d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:16.711720 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6315ed7-17c8-4bb0-9fcb-73aabe6c596d service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:16.730470 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 665/1340] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:11:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:11:20.719343 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65651f6-04b7-4c69-9cb7-6acee38b5438 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:20.719916 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65651f6-04b7-4c69-9cb7-6acee38b5438 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:20.719916 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65651f6-04b7-4c69-9cb7-6acee38b5438 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:20.721167 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 676/1341] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:11:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:20.726497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062db6a8-78d5-453c-81a5-b04b929a5037 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:20.726979 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062db6a8-78d5-453c-81a5-b04b929a5037 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:20.727075 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062db6a8-78d5-453c-81a5-b04b929a5037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:20.799233 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-062db6a8-78d5-453c-81a5-b04b929a5037 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:20.800475 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-062db6a8-78d5-453c-81a5-b04b929a5037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:20.886631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:20.898976 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 666/1342] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:11:21.193866 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65651f6-04b7-4c69-9cb7-6acee38b5438 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:21.195512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f65651f6-04b7-4c69-9cb7-6acee38b5438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:21.305536 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65651f6-04b7-4c69-9cb7-6acee38b5438 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:21.345567 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:21.383849 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97751656-4d7d-4775-973b-3dd7b92a41bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:21.384458 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97751656-4d7d-4775-973b-3dd7b92a41bc service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:21.384732 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97751656-4d7d-4775-973b-3dd7b92a41bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:21.384998 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97751656-4d7d-4775-973b-3dd7b92a41bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:21.387099 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-97751656-4d7d-4775-973b-3dd7b92a41bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:21.498224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97751656-4d7d-4775-973b-3dd7b92a41bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:21.500023 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97751656-4d7d-4775-973b-3dd7b92a41bc service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:21.519119 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 677/1343] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:11:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:11:24.117606 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084dbae-60d9-495d-abbe-ab601f70fac2 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:24.118160 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084dbae-60d9-495d-abbe-ab601f70fac2 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:24.118160 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084dbae-60d9-495d-abbe-ab601f70fac2 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:24.119499 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 667/1344] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:24.125874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4713af6-0d2a-444e-8ccd-8de3bd72a3d2 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:24.126441 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4713af6-0d2a-444e-8ccd-8de3bd72a3d2 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:24.126441 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4713af6-0d2a-444e-8ccd-8de3bd72a3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:24.202548 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-d4713af6-0d2a-444e-8ccd-8de3bd72a3d2 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:24.203811 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d4713af6-0d2a-444e-8ccd-8de3bd72a3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:24.226618 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:24.316473 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 678/1345] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:11:24.615555 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084dbae-60d9-495d-abbe-ab601f70fac2 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:24.617099 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-1084dbae-60d9-495d-abbe-ab601f70fac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:24.724342 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084dbae-60d9-495d-abbe-ab601f70fac2 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:24.803958 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551b68ae-91a8-4284-ae55-66b3b8374306 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:24.804570 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b68ae-91a8-4284-ae55-66b3b8374306 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:24.804847 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b68ae-91a8-4284-ae55-66b3b8374306 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:24.805123 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b68ae-91a8-4284-ae55-66b3b8374306 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:24.806786 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-551b68ae-91a8-4284-ae55-66b3b8374306 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:24.852225 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:24.926625 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551b68ae-91a8-4284-ae55-66b3b8374306 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:24.928215 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551b68ae-91a8-4284-ae55-66b3b8374306 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:24.947786 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 668/1346] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:11:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:11:29.163129 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273382a-f63f-4c82-a66f-14308956983c None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:29.163677 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273382a-f63f-4c82-a66f-14308956983c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:29.163954 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273382a-f63f-4c82-a66f-14308956983c None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:29.165908 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 679/1347] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:11:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:29.173800 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c82a5e-8a8e-40ed-a1d9-991cfcb63953 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:29.174606 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c82a5e-8a8e-40ed-a1d9-991cfcb63953 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:29.175043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c82a5e-8a8e-40ed-a1d9-991cfcb63953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:29.262402 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-c7c82a5e-8a8e-40ed-a1d9-991cfcb63953 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:29.263681 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-c7c82a5e-8a8e-40ed-a1d9-991cfcb63953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:29.384715 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:29.395440 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 669/1348] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:11:29.677519 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0273382a-f63f-4c82-a66f-14308956983c None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:29.679358 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-0273382a-f63f-4c82-a66f-14308956983c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:29.794554 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0273382a-f63f-4c82-a66f-14308956983c None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:29.845827 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:29.873473 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd77d6fa-414b-4436-b6fc-567c3faabbb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:29.874078 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77d6fa-414b-4436-b6fc-567c3faabbb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:29.874354 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77d6fa-414b-4436-b6fc-567c3faabbb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:29.874623 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77d6fa-414b-4436-b6fc-567c3faabbb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:29.876253 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-bd77d6fa-414b-4436-b6fc-567c3faabbb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:29.986476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd77d6fa-414b-4436-b6fc-567c3faabbb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:29.987991 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd77d6fa-414b-4436-b6fc-567c3faabbb8 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:30.006512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 680/1349] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:11:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:11:34.998193 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d59fa6-fc42-4f5d-9051-0ed8f928e11c None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:34.998688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d59fa6-fc42-4f5d-9051-0ed8f928e11c None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:34.998783 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d59fa6-fc42-4f5d-9051-0ed8f928e11c None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:35.000829 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 670/1350] 10.222.0.16 () {54 vars in 921 bytes} [Mon Sep 5 12:11:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:35.006753 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd2de0-7ab4-4ac7-ac40-90a72d1b0dc3 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:35.007238 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd2de0-7ab4-4ac7-ac40-90a72d1b0dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:35.007488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd2de0-7ab4-4ac7-ac40-90a72d1b0dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:35.109160 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-a6fd2de0-7ab4-4ac7-ac40-90a72d1b0dc3 None None] MFA Rules not processed for user `524bcfffcdfa4790885e0d4cbd66fa80`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:35.110490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a6fd2de0-7ab4-4ac7-ac40-90a72d1b0dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:35.134075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:35.287435 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 681/1351] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Sep 5 12:11:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:11:39.636621 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06bde0-dedc-410e-a889-75d37ce59fbb None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:39.637134 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06bde0-dedc-410e-a889-75d37ce59fbb None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:39.637542 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06bde0-dedc-410e-a889-75d37ce59fbb None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:39.639059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 671/1352] 10.222.0.16 () {54 vars in 921 bytes} [Mon Sep 5 12:11:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:39.644809 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7122db-7be2-452c-b1ae-fe5b42f43549 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:39.645303 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7122db-7be2-452c-b1ae-fe5b42f43549 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:39.645396 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7122db-7be2-452c-b1ae-fe5b42f43549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:39.733085 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-4e7122db-7be2-452c-b1ae-fe5b42f43549 None None] MFA Rules not processed for user `524bcfffcdfa4790885e0d4cbd66fa80`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:39.734418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4e7122db-7be2-452c-b1ae-fe5b42f43549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:39.815986 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:39.856660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 682/1353] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Sep 5 12:11:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:11:41.259035 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75fbbdd-7372-4866-ab4a-77a103196003 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:41.259570 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75fbbdd-7372-4866-ab4a-77a103196003 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:41.259699 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75fbbdd-7372-4866-ab4a-77a103196003 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:41.261045 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 672/1354] 10.222.0.16 () {54 vars in 921 bytes} [Mon Sep 5 12:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:41.266806 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c319d-c0db-4551-b32b-e79d20c6ccac None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:41.267293 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c319d-c0db-4551-b32b-e79d20c6ccac None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:41.267414 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c319d-c0db-4551-b32b-e79d20c6ccac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:41.349136 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-254c319d-c0db-4551-b32b-e79d20c6ccac None None] MFA Rules not processed for user `524bcfffcdfa4790885e0d4cbd66fa80`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:41.350350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-254c319d-c0db-4551-b32b-e79d20c6ccac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:41.467147 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 683/1355] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Sep 5 12:11:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:11:44.881165 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e541045-8312-4df2-bff8-07e6e0f59a36 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:44.881784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e541045-8312-4df2-bff8-07e6e0f59a36 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:44.881784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e541045-8312-4df2-bff8-07e6e0f59a36 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:44.883080 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 673/1356] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:44.889172 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e6af7-f63a-4795-8d42-79bb4d4e3baf None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:44.889624 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e6af7-f63a-4795-8d42-79bb4d4e3baf None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:44.889742 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e6af7-f63a-4795-8d42-79bb4d4e3baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:44.925833 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:44.964660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-296e6af7-f63a-4795-8d42-79bb4d4e3baf None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:44.965893 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-296e6af7-f63a-4795-8d42-79bb4d4e3baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:45.089832 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 684/1357] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:11:45.400698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ad17a-0945-4ebb-a71d-928474933a95 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:45.400984 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ad17a-0945-4ebb-a71d-928474933a95 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:45.401267 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ad17a-0945-4ebb-a71d-928474933a95 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:45.402620 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 674/1358] 10.222.0.16 () {54 vars in 962 bytes} [Mon Sep 5 12:11:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:45.412048 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2f702-42fc-4467-b0d4-9d91f869d1ca None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:45.412998 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2f702-42fc-4467-b0d4-9d91f869d1ca None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:45.413451 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2f702-42fc-4467-b0d4-9d91f869d1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:45.509322 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-daf2f702-42fc-4467-b0d4-9d91f869d1ca None None] MFA Rules not processed for user `7cda290418504f0db85d665479a05353`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:45.510758 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-daf2f702-42fc-4467-b0d4-9d91f869d1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:45.570707 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:45.644093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 685/1359] 10.222.0.16 () {58 vars in 1059 bytes} [Mon Sep 5 12:11:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:11:45.651781 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ad17a-0945-4ebb-a71d-928474933a95 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:45.653886 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-7f3ad17a-0945-4ebb-a71d-928474933a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:45.813628 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ad17a-0945-4ebb-a71d-928474933a95 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:45.909889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:45.956300 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645eefce-0b14-48eb-be1a-0a6efa6b5211 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:45.957329 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645eefce-0b14-48eb-be1a-0a6efa6b5211 service nova] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:45.957613 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645eefce-0b14-48eb-be1a-0a6efa6b5211 service nova] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:45.957883 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645eefce-0b14-48eb-be1a-0a6efa6b5211 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:45.959676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-645eefce-0b14-48eb-be1a-0a6efa6b5211 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:46.059156 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645eefce-0b14-48eb-be1a-0a6efa6b5211 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:46.061012 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645eefce-0b14-48eb-be1a-0a6efa6b5211 service nova] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:46.083882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 675/1360] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:11:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:11:48.246936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323db81-1923-47de-8eab-f491cebe668f None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:48.246936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323db81-1923-47de-8eab-f491cebe668f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:48.247659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323db81-1923-47de-8eab-f491cebe668f None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:48.249771 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 686/1361] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:11:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:48.255109 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e48269-7c16-46e0-b7ef-d7cc9a40e764 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:48.255636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e48269-7c16-46e0-b7ef-d7cc9a40e764 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:48.255920 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e48269-7c16-46e0-b7ef-d7cc9a40e764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:48.333059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-60e48269-7c16-46e0-b7ef-d7cc9a40e764 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:48.334280 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-60e48269-7c16-46e0-b7ef-d7cc9a40e764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:48.385779 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:48.454882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 676/1362] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:11:48.763353 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323db81-1923-47de-8eab-f491cebe668f None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:48.764939 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d323db81-1923-47de-8eab-f491cebe668f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:48.944682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323db81-1923-47de-8eab-f491cebe668f None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:48.955572 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:49.073479 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e5d2c-9e65-4afd-905b-db17a7b195c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:49.074628 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e5d2c-9e65-4afd-905b-db17a7b195c9 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:49.075204 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e5d2c-9e65-4afd-905b-db17a7b195c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:49.075483 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e5d2c-9e65-4afd-905b-db17a7b195c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:49.078017 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d10e5d2c-9e65-4afd-905b-db17a7b195c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:49.186048 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10e5d2c-9e65-4afd-905b-db17a7b195c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:49.187739 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10e5d2c-9e65-4afd-905b-db17a7b195c9 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:49.208537 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 687/1363] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:11:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:11:51.488007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a702dc-6242-4acf-9b3d-2d22be5c2422 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:51.488878 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a702dc-6242-4acf-9b3d-2d22be5c2422 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:51.489155 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a702dc-6242-4acf-9b3d-2d22be5c2422 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:51.490512 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 677/1364] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:11:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:51.497011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a87f4-20c0-4ff4-beb8-a168e7a43eca None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:51.497011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a87f4-20c0-4ff4-beb8-a168e7a43eca None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:51.497011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a87f4-20c0-4ff4-beb8-a168e7a43eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:51.521205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:51.584435 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-1a8a87f4-20c0-4ff4-beb8-a168e7a43eca None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:51.586072 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1a8a87f4-20c0-4ff4-beb8-a168e7a43eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:51.705133 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 688/1365] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:11:52.001011 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ed7ea-0a1f-4d04-a0fd-e35f7391c03b None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:52.001631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ed7ea-0a1f-4d04-a0fd-e35f7391c03b None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:52.002106 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ed7ea-0a1f-4d04-a0fd-e35f7391c03b None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:52.003654 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 678/1366] 10.222.0.16 () {54 vars in 962 bytes} [Mon Sep 5 12:11:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:11:52.009535 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82889a53-5ace-4d1a-97ea-493e4a213e11 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:52.009844 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82889a53-5ace-4d1a-97ea-493e4a213e11 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:52.010117 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82889a53-5ace-4d1a-97ea-493e4a213e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:52.093091 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-82889a53-5ace-4d1a-97ea-493e4a213e11 None None] MFA Rules not processed for user `7cda290418504f0db85d665479a05353`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:52.094293 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-82889a53-5ace-4d1a-97ea-493e4a213e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:52.131392 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:52.226160 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 689/1367] 10.222.0.16 () {58 vars in 1059 bytes} [Mon Sep 5 12:11:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:11:52.232617 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ed7ea-0a1f-4d04-a0fd-e35f7391c03b None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:52.234053 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-a27ed7ea-0a1f-4d04-a0fd-e35f7391c03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:52.418943 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ed7ea-0a1f-4d04-a0fd-e35f7391c03b None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:52.443164 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:52.559908 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e0aea0-b653-4966-a4d9-2ebecd92a206 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:52.560609 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0aea0-b653-4966-a4d9-2ebecd92a206 service nova] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:52.560912 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0aea0-b653-4966-a4d9-2ebecd92a206 service nova] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:52.561044 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0aea0-b653-4966-a4d9-2ebecd92a206 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:52.562697 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-89e0aea0-b653-4966-a4d9-2ebecd92a206 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:52.662009 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e0aea0-b653-4966-a4d9-2ebecd92a206 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:52.663608 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e0aea0-b653-4966-a4d9-2ebecd92a206 service nova] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:52.684919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 679/1368] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:11:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:11:53.294108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7997b7-0efd-4d81-931e-62502af9f6f5 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:53.295041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7997b7-0efd-4d81-931e-62502af9f6f5 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:53.295552 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7997b7-0efd-4d81-931e-62502af9f6f5 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:53.297081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 690/1369] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:11:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:53.302495 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079bf4a-a7fd-48fd-a6ed-3a0d48fcd1d2 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:53.302778 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079bf4a-a7fd-48fd-a6ed-3a0d48fcd1d2 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:53.303052 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079bf4a-a7fd-48fd-a6ed-3a0d48fcd1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:53.339328 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:53.382164 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-c079bf4a-a7fd-48fd-a6ed-3a0d48fcd1d2 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:53.383403 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-c079bf4a-a7fd-48fd-a6ed-3a0d48fcd1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:53.487871 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 680/1370] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:11:54.086395 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2dc48d-9260-4a5e-9be6-23eaa6cb3fd3 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:54.086714 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2dc48d-9260-4a5e-9be6-23eaa6cb3fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:54.086975 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2dc48d-9260-4a5e-9be6-23eaa6cb3fd3 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:54.088626 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 691/1371] 10.222.0.16 () {54 vars in 964 bytes} [Mon Sep 5 12:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:54.094071 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f7b3b-0556-4e7a-948d-9da5ca23d20a None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:54.094565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f7b3b-0556-4e7a-948d-9da5ca23d20a None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:54.094856 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f7b3b-0556-4e7a-948d-9da5ca23d20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:54.180302 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-945f7b3b-0556-4e7a-948d-9da5ca23d20a None None] MFA Rules not processed for user `e8a47a0d9bfd473cbf0147d7801442d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:54.181561 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-945f7b3b-0556-4e7a-948d-9da5ca23d20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:54.235927 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:54.268402 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 681/1372] 10.222.0.16 () {58 vars in 1061 bytes} [Mon Sep 5 12:11:54 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:11:54.276191 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2dc48d-9260-4a5e-9be6-23eaa6cb3fd3 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:54.278526 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6e2dc48d-9260-4a5e-9be6-23eaa6cb3fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:54.397544 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2dc48d-9260-4a5e-9be6-23eaa6cb3fd3 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:54.442121 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:54.481364 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034cb250-0a32-411f-aaf0-7fe74344f463 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:54.482027 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034cb250-0a32-411f-aaf0-7fe74344f463 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:54.482226 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034cb250-0a32-411f-aaf0-7fe74344f463 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:54.482491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034cb250-0a32-411f-aaf0-7fe74344f463 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:54.484142 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-034cb250-0a32-411f-aaf0-7fe74344f463 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:54.609155 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034cb250-0a32-411f-aaf0-7fe74344f463 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:54.610424 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034cb250-0a32-411f-aaf0-7fe74344f463 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:54.631164 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 692/1373] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:11:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:11:54.779011 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c13510-e9cc-49ab-b884-53f20ddfda92 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:54.779304 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c13510-e9cc-49ab-b884-53f20ddfda92 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:54.779622 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c13510-e9cc-49ab-b884-53f20ddfda92 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:54.780980 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 682/1374] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:54.792770 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c6257-3736-49c0-ad7c-fa3d34410ce6 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:54.793189 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c6257-3736-49c0-ad7c-fa3d34410ce6 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:54.793372 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c6257-3736-49c0-ad7c-fa3d34410ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:54.873375 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-8e4c6257-3736-49c0-ad7c-fa3d34410ce6 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:54.874522 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8e4c6257-3736-49c0-ad7c-fa3d34410ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:54.898617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:54.978588 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 693/1375] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:11:55.269816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c13510-e9cc-49ab-b884-53f20ddfda92 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:55.271675 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-f4c13510-e9cc-49ab-b884-53f20ddfda92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:55.448531 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c13510-e9cc-49ab-b884-53f20ddfda92 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:55.503084 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:55.577506 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc23ff2-003a-46c8-882f-1a0679ad3045 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:55.578113 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc23ff2-003a-46c8-882f-1a0679ad3045 service nova] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:55.578393 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc23ff2-003a-46c8-882f-1a0679ad3045 service nova] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:55.578664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc23ff2-003a-46c8-882f-1a0679ad3045 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:55.580327 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-bbc23ff2-003a-46c8-882f-1a0679ad3045 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:55.678144 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc23ff2-003a-46c8-882f-1a0679ad3045 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:55.679784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc23ff2-003a-46c8-882f-1a0679ad3045 service nova] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:55.700647 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 683/1376] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:11:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:11:57.687798 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316028d-0912-43ca-8970-ae098ac8544c None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:57.690398 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316028d-0912-43ca-8970-ae098ac8544c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:57.690398 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316028d-0912-43ca-8970-ae098ac8544c None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:57.690398 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 694/1377] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:11:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:57.697926 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ada1b-4066-4e14-ad35-93479ba9e185 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:57.697926 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ada1b-4066-4e14-ad35-93479ba9e185 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:57.697926 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ada1b-4066-4e14-ad35-93479ba9e185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:57.806275 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-a59ada1b-4066-4e14-ad35-93479ba9e185 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:57.808021 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-a59ada1b-4066-4e14-ad35-93479ba9e185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:57.820026 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:57.918503 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 684/1378] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:11:58.233993 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1316028d-0912-43ca-8970-ae098ac8544c None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:58.234885 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1316028d-0912-43ca-8970-ae098ac8544c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:58.435109 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1316028d-0912-43ca-8970-ae098ac8544c None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:58.439678 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:58.581501 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9affa80-e992-423c-8f76-d7618344119d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:58.582521 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9affa80-e992-423c-8f76-d7618344119d service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:58.583206 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9affa80-e992-423c-8f76-d7618344119d service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:58.583745 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9affa80-e992-423c-8f76-d7618344119d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:58.586095 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e9affa80-e992-423c-8f76-d7618344119d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:58.692334 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9affa80-e992-423c-8f76-d7618344119d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:58.695090 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9affa80-e992-423c-8f76-d7618344119d service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:58.718333 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 695/1379] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:11:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:12:00.841388 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce92c7b-0455-4af0-92b8-a6170fb15f33 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:00.842048 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce92c7b-0455-4af0-92b8-a6170fb15f33 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:00.842048 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce92c7b-0455-4af0-92b8-a6170fb15f33 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:00.847952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 685/1380] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:12:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:12:00.857313 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090818a-f789-4ada-90cd-1935174ed2bd None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:00.857881 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090818a-f789-4ada-90cd-1935174ed2bd None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:00.857881 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090818a-f789-4ada-90cd-1935174ed2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:00.869815 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:00.962883 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-e090818a-f789-4ada-90cd-1935174ed2bd None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:12:00.964046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e090818a-f789-4ada-90cd-1935174ed2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:01.143934 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 696/1381] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:12:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:12:01.440525 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce92c7b-0455-4af0-92b8-a6170fb15f33 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:12:01.442068 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-1ce92c7b-0455-4af0-92b8-a6170fb15f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:01.626679 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:01.640150 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce92c7b-0455-4af0-92b8-a6170fb15f33 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:12:01.793464 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031c1f24-9b40-439f-b68f-7454616dc3d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:12:01.794568 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c1f24-9b40-439f-b68f-7454616dc3d0 service nova] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:01.795095 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c1f24-9b40-439f-b68f-7454616dc3d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:01.795623 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c1f24-9b40-439f-b68f-7454616dc3d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:01.797585 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-031c1f24-9b40-439f-b68f-7454616dc3d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:01.914119 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031c1f24-9b40-439f-b68f-7454616dc3d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:12:01.915872 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031c1f24-9b40-439f-b68f-7454616dc3d0 service nova] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:12:01.928335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:01.947807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 686/1382] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:12:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:12:04.209451 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5090901-1fdf-46b5-90eb-a56eb9a46305 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:04.210187 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5090901-1fdf-46b5-90eb-a56eb9a46305 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:04.210187 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5090901-1fdf-46b5-90eb-a56eb9a46305 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:04.211398 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 697/1383] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:12:04.216776 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1acd84c-e76c-44a9-86b8-194bad546fbb None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:04.217293 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1acd84c-e76c-44a9-86b8-194bad546fbb None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:04.217293 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1acd84c-e76c-44a9-86b8-194bad546fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:04.289541 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-c1acd84c-e76c-44a9-86b8-194bad546fbb None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:12:04.290759 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-c1acd84c-e76c-44a9-86b8-194bad546fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:04.412483 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 687/1384] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:12:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:12:04.709089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5090901-1fdf-46b5-90eb-a56eb9a46305 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:12:04.713127 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f5090901-1fdf-46b5-90eb-a56eb9a46305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:04.879460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:04.948403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5090901-1fdf-46b5-90eb-a56eb9a46305 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:12:05.083074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c49501-285b-4ae1-8c77-8e675a8ab482 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:12:05.083863 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c49501-285b-4ae1-8c77-8e675a8ab482 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:05.084210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c49501-285b-4ae1-8c77-8e675a8ab482 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:05.084615 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c49501-285b-4ae1-8c77-8e675a8ab482 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:05.086695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-16c49501-285b-4ae1-8c77-8e675a8ab482 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:05.147648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:05.192362 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c49501-285b-4ae1-8c77-8e675a8ab482 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:12:05.194041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c49501-285b-4ae1-8c77-8e675a8ab482 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:12:05.213945 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 698/1385] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:12:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:12:07.207925 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f4e20-8d90-4c1c-a9ae-e09e976e9bef None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:07.208457 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f4e20-8d90-4c1c-a9ae-e09e976e9bef None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:07.208457 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f4e20-8d90-4c1c-a9ae-e09e976e9bef None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:07.209797 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 688/1386] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:12:07.215107 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cbd549-d9a9-4295-87c8-60191e242b3e None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:07.215573 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cbd549-d9a9-4295-87c8-60191e242b3e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:07.215672 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cbd549-d9a9-4295-87c8-60191e242b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:07.285424 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-68cbd549-d9a9-4295-87c8-60191e242b3e None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:12:07.286658 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-68cbd549-d9a9-4295-87c8-60191e242b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:07.392417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 699/1387] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:12:07.687287 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f4e20-8d90-4c1c-a9ae-e09e976e9bef None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:12:07.688955 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-409f4e20-8d90-4c1c-a9ae-e09e976e9bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:07.785444 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:07.860506 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f4e20-8d90-4c1c-a9ae-e09e976e9bef None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:12:07.998798 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58043a71-44d0-4f2b-a3e3-0a940d9939e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:12:07.999490 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58043a71-44d0-4f2b-a3e3-0a940d9939e6 service nova] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:07.999829 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58043a71-44d0-4f2b-a3e3-0a940d9939e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:08.000167 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58043a71-44d0-4f2b-a3e3-0a940d9939e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:08.002064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-58043a71-44d0-4f2b-a3e3-0a940d9939e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:08.049059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:08.105427 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58043a71-44d0-4f2b-a3e3-0a940d9939e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:12:08.107232 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58043a71-44d0-4f2b-a3e3-0a940d9939e6 service nova] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:12:08.127421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 689/1388] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:12:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:12:10.047325 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca921d-ae5a-4ef3-9750-ad45ad086e00 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:10.047974 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca921d-ae5a-4ef3-9750-ad45ad086e00 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:10.047974 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca921d-ae5a-4ef3-9750-ad45ad086e00 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:10.049255 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 700/1389] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:12:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:12:10.054719 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43514a-788d-4f73-a9e5-a62e79e68762 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:10.055198 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43514a-788d-4f73-a9e5-a62e79e68762 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:10.055305 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43514a-788d-4f73-a9e5-a62e79e68762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:10.126170 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-7b43514a-788d-4f73-a9e5-a62e79e68762 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:12:10.127390 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-7b43514a-788d-4f73-a9e5-a62e79e68762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:10.130375 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675976c-3812-481e-813f-a4238fc1b527 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:10.130662 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675976c-3812-481e-813f-a4238fc1b527 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:10.130933 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675976c-3812-481e-813f-a4238fc1b527 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:10.132557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 701/1390] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:12:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:12:10.138041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339f76d-4efc-43b9-b340-438fa752001f None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:10.138325 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339f76d-4efc-43b9-b340-438fa752001f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:10.138602 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339f76d-4efc-43b9-b340-438fa752001f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:10.176410 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:10.208153 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-e339f76d-4efc-43b9-b340-438fa752001f None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:12:10.209358 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e339f76d-4efc-43b9-b340-438fa752001f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:10.227174 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 690/1391] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:12:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:12:10.310551 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 702/1392] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:12:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:12:10.525005 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197fe8d-887b-4502-91de-cef389d744f5 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:10.525290 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197fe8d-887b-4502-91de-cef389d744f5 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:10.525567 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197fe8d-887b-4502-91de-cef389d744f5 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:10.527386 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 691/1393] 10.222.0.16 () {54 vars in 961 bytes} [Mon Sep 5 12:12:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:12:10.532931 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6891c-1533-40c4-b8d6-f43dae51862a None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:10.533211 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6891c-1533-40c4-b8d6-f43dae51862a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:10.533480 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6891c-1533-40c4-b8d6-f43dae51862a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:10.600883 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197fe8d-887b-4502-91de-cef389d744f5 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:12:10.602416 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-1197fe8d-887b-4502-91de-cef389d744f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:10.614121 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-fdf6891c-1533-40c4-b8d6-f43dae51862a None None] MFA Rules not processed for user `ed75c0be809d4a0eb0e2ccc59805b1ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:12:10.615343 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-fdf6891c-1533-40c4-b8d6-f43dae51862a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:10.617019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:10.665875 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:10.694231 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 703/1394] 10.222.0.16 () {58 vars in 1058 bytes} [Mon Sep 5 12:12:10 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:12:10.702801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6891c-1533-40c4-b8d6-f43dae51862a None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:12:10.710211 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6891c-1533-40c4-b8d6-f43dae51862a None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:12:10.755580 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197fe8d-887b-4502-91de-cef389d744f5 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:12:10.790841 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af7667-7ac6-4f2c-8372-9730504e1e19 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed75c0be809d4a0eb0e2ccc59805b1ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:12:10.791441 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af7667-7ac6-4f2c-8372-9730504e1e19 service glance] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:10.791745 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af7667-7ac6-4f2c-8372-9730504e1e19 service glance] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:10.792015 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af7667-7ac6-4f2c-8372-9730504e1e19 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:10.794607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-82af7667-7ac6-4f2c-8372-9730504e1e19 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:10.854483 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:10.881838 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8184c215-96b2-4ad1-b9d4-c10a7e6b2d4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:12:10.882440 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184c215-96b2-4ad1-b9d4-c10a7e6b2d4c service nova] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:10.882710 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184c215-96b2-4ad1-b9d4-c10a7e6b2d4c service nova] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:10.882985 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184c215-96b2-4ad1-b9d4-c10a7e6b2d4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:10.884630 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-8184c215-96b2-4ad1-b9d4-c10a7e6b2d4c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:10.926220 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af7667-7ac6-4f2c-8372-9730504e1e19 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:12:10.927695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af7667-7ac6-4f2c-8372-9730504e1e19 service glance] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:12:10.948680 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:10.955020 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 704/1395] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:12:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:12:10.984848 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8184c215-96b2-4ad1-b9d4-c10a7e6b2d4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:12:10.986459 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8184c215-96b2-4ad1-b9d4-c10a7e6b2d4c service nova] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:12:11.004898 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 692/1396] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:12:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:12:11.717711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1afca-8779-4bf7-95ae-7dd7863880bf None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:11.718200 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1afca-8779-4bf7-95ae-7dd7863880bf None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:11.718292 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1afca-8779-4bf7-95ae-7dd7863880bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:11.787319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-98d1afca-8779-4bf7-95ae-7dd7863880bf None None] MFA Rules not processed for user `8c5ae98336c441118d01f46a50f732d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:12:11.788569 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-98d1afca-8779-4bf7-95ae-7dd7863880bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:11.876253 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 705/1397] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Sep 5 12:12:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:12:11.926890 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a46098-cfd5-4349-9667-f5dc163596a5 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:11.927411 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a46098-cfd5-4349-9667-f5dc163596a5 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:11.927585 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a46098-cfd5-4349-9667-f5dc163596a5 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:11.929198 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 693/1398] 10.222.0.16 () {54 vars in 943 bytes} [Mon Sep 5 12:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:12:11.935477 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6163c1c-b71f-4b24-9767-bae7b1df9afa None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:11.935795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6163c1c-b71f-4b24-9767-bae7b1df9afa None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:11.936067 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6163c1c-b71f-4b24-9767-bae7b1df9afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:12.132985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-a6163c1c-b71f-4b24-9767-bae7b1df9afa None None] MFA Rules not processed for user `857eb7657e7d45938baca17f91a16901`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:12:12.134239 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a6163c1c-b71f-4b24-9767-bae7b1df9afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:12.167997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:12.246999 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 706/1399] 10.222.0.16 () {58 vars in 1040 bytes} [Mon Sep 5 12:12:11 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:12:12.256299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a46098-cfd5-4349-9667-f5dc163596a5 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:12:12.257889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-52a46098-cfd5-4349-9667-f5dc163596a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:12.360590 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:12.415642 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a46098-cfd5-4349-9667-f5dc163596a5 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:12:12.520716 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68f403-7487-49f4-8377-8e682d76f5ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '857eb7657e7d45938baca17f91a16901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:12:12.521234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68f403-7487-49f4-8377-8e682d76f5ef service swift] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:12.521515 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68f403-7487-49f4-8377-8e682d76f5ef service swift] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:12.521787 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68f403-7487-49f4-8377-8e682d76f5ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:12.523431 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-ca68f403-7487-49f4-8377-8e682d76f5ef service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:12.608665 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca68f403-7487-49f4-8377-8e682d76f5ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:12:12.610253 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca68f403-7487-49f4-8377-8e682d76f5ef service swift] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:12:12.613974 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:12.620511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 694/1400] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Sep 5 12:12:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 12:12:12.994900 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2baff-9911-49b2-a4c9-4f2f8a534684 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:12.996026 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2baff-9911-49b2-a4c9-4f2f8a534684 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:12.996133 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2baff-9911-49b2-a4c9-4f2f8a534684 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:12.997798 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 707/1401] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:12:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:12:13.003354 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8f0fb-fdae-48bf-b066-c2d6f1af21d2 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:13.003935 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8f0fb-fdae-48bf-b066-c2d6f1af21d2 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:13.004454 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8f0fb-fdae-48bf-b066-c2d6f1af21d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:13.079067 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-78e8f0fb-fdae-48bf-b066-c2d6f1af21d2 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:12:13.080545 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-78e8f0fb-fdae-48bf-b066-c2d6f1af21d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:13.187239 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 695/1402] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:12:13.478152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2baff-9911-49b2-a4c9-4f2f8a534684 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:12:13.480393 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1ed2baff-9911-49b2-a4c9-4f2f8a534684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:13.641290 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2baff-9911-49b2-a4c9-4f2f8a534684 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:12:13.644181 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:13.767814 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43adc6f6-0862-4278-978a-0431f193eb74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:12:13.768425 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43adc6f6-0862-4278-978a-0431f193eb74 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:13.768701 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43adc6f6-0862-4278-978a-0431f193eb74 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:13.768971 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43adc6f6-0862-4278-978a-0431f193eb74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:13.770748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-43adc6f6-0862-4278-978a-0431f193eb74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:13.874005 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43adc6f6-0862-4278-978a-0431f193eb74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:12:13.875859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43adc6f6-0862-4278-978a-0431f193eb74 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:12:13.897584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 708/1403] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:12:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:12:15.933783 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780b084-6ed5-4a8d-99c2-dd06f90265d6 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:15.934331 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780b084-6ed5-4a8d-99c2-dd06f90265d6 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:15.934331 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780b084-6ed5-4a8d-99c2-dd06f90265d6 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:15.935750 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 696/1404] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:12:15.941229 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798720d0-29aa-4fab-88cf-f7f98dd65eb0 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:15.941705 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798720d0-29aa-4fab-88cf-f7f98dd65eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:15.941799 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798720d0-29aa-4fab-88cf-f7f98dd65eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:15.952005 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:16.015621 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-798720d0-29aa-4fab-88cf-f7f98dd65eb0 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:12:16.016800 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-798720d0-29aa-4fab-88cf-f7f98dd65eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:16.126355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 709/1405] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:12:16.420615 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3780b084-6ed5-4a8d-99c2-dd06f90265d6 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:12:16.422231 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-3780b084-6ed5-4a8d-99c2-dd06f90265d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:16.509772 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:16.585701 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3780b084-6ed5-4a8d-99c2-dd06f90265d6 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:12:16.714837 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d0aec-4989-4108-b10f-a628990a821d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:12:16.715404 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d0aec-4989-4108-b10f-a628990a821d service nova] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:16.715738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d0aec-4989-4108-b10f-a628990a821d service nova] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:16.716018 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d0aec-4989-4108-b10f-a628990a821d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:16.717665 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-321d0aec-4989-4108-b10f-a628990a821d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:16.765534 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:16.825674 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d0aec-4989-4108-b10f-a628990a821d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:12:16.827474 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d0aec-4989-4108-b10f-a628990a821d service nova] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:12:16.861791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 697/1406] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:12:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:12:18.865287 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc38fa-2431-4566-b0d8-f49967d21d95 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:18.865851 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc38fa-2431-4566-b0d8-f49967d21d95 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:18.865851 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc38fa-2431-4566-b0d8-f49967d21d95 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:18.867319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 710/1407] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:12:18.873248 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f050a83-d279-448b-b75d-06b22688767c None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:18.873699 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f050a83-d279-448b-b75d-06b22688767c None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:18.873784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f050a83-d279-448b-b75d-06b22688767c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:18.953406 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-7f050a83-d279-448b-b75d-06b22688767c None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:12:18.954636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-7f050a83-d279-448b-b75d-06b22688767c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:19.081343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 698/1408] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:12:19.387470 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc38fa-2431-4566-b0d8-f49967d21d95 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:12:19.389225 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8cbc38fa-2431-4566-b0d8-f49967d21d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:19.506407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:19.583938 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc38fa-2431-4566-b0d8-f49967d21d95 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:12:19.724835 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8eacf-12bb-40b8-8637-ec5db3ca3046 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:12:19.725358 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8eacf-12bb-40b8-8637-ec5db3ca3046 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:19.725637 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8eacf-12bb-40b8-8637-ec5db3ca3046 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:19.725906 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8eacf-12bb-40b8-8637-ec5db3ca3046 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:19.727616 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-21e8eacf-12bb-40b8-8637-ec5db3ca3046 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:19.794905 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:19.844263 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e8eacf-12bb-40b8-8637-ec5db3ca3046 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:12:19.846369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e8eacf-12bb-40b8-8637-ec5db3ca3046 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:12:19.864922 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 711/1409] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:12:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:12:29.078481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=93 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:29.862336 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=24 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:34.692242 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f49f30-7102-4f39-ba4f-2f51b42d9fdf None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:34.692781 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f49f30-7102-4f39-ba4f-2f51b42d9fdf None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:34.692781 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f49f30-7102-4f39-ba4f-2f51b42d9fdf None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:34.694102 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 699/1410] 10.222.0.16 () {54 vars in 918 bytes} [Mon Sep 5 12:12:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:12:34.699377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e8694-3298-49f5-9d66-7ec8c6977fbc None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:34.699865 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e8694-3298-49f5-9d66-7ec8c6977fbc None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:34.700000 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e8694-3298-49f5-9d66-7ec8c6977fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:34.775787 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-9a0e8694-3298-49f5-9d66-7ec8c6977fbc None None] MFA Rules not processed for user `524bcfffcdfa4790885e0d4cbd66fa80`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:12:34.777007 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9a0e8694-3298-49f5-9d66-7ec8c6977fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:34.882848 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 712/1411] 10.222.0.16 () {58 vars in 1015 bytes} [Mon Sep 5 12:12:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:12:34.897092 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62537c-e1fb-4618-ba8e-54ca45f5815f None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:34.897380 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62537c-e1fb-4618-ba8e-54ca45f5815f None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:34.897658 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62537c-e1fb-4618-ba8e-54ca45f5815f None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:34.899020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 700/1412] 10.222.0.16 () {54 vars in 918 bytes} [Mon Sep 5 12:12:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:12:34.903756 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5edeed-2525-4334-b9d8-6d3064980f26 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:34.904070 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5edeed-2525-4334-b9d8-6d3064980f26 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:34.904344 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5edeed-2525-4334-b9d8-6d3064980f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:34.977184 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:34.980469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-4f5edeed-2525-4334-b9d8-6d3064980f26 None None] MFA Rules not processed for user `7cda290418504f0db85d665479a05353`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:12:34.982224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4f5edeed-2525-4334-b9d8-6d3064980f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:35.100864 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 713/1413] 10.222.0.16 () {58 vars in 1015 bytes} [Mon Sep 5 12:12:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:12:37.500039 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b737e3-0cd6-4b2b-ac6d-d17dc72f0dc2 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:37.500574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b737e3-0cd6-4b2b-ac6d-d17dc72f0dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:37.500574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b737e3-0cd6-4b2b-ac6d-d17dc72f0dc2 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:37.501859 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 701/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 12:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:12:37.507271 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16348032-4ea5-4773-a7a4-6c65e3ea45ff None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:37.507758 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16348032-4ea5-4773-a7a4-6c65e3ea45ff None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:37.507860 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16348032-4ea5-4773-a7a4-6c65e3ea45ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:37.578735 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-16348032-4ea5-4773-a7a4-6c65e3ea45ff None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:12:37.580005 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-16348032-4ea5-4773-a7a4-6c65e3ea45ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:37.635035 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:37.677722 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 714/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 12:12:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:12:37.970799 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b737e3-0cd6-4b2b-ac6d-d17dc72f0dc2 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:12:37.972451 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-f6b737e3-0cd6-4b2b-ac6d-d17dc72f0dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:38.121915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b737e3-0cd6-4b2b-ac6d-d17dc72f0dc2 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:12:38.198009 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:38.238401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1244d964-b936-4bb5-a765-154c10ad5c61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:12:38.238911 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244d964-b936-4bb5-a765-154c10ad5c61 service nova] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:38.239187 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244d964-b936-4bb5-a765-154c10ad5c61 service nova] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:38.239456 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244d964-b936-4bb5-a765-154c10ad5c61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:38.241398 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-1244d964-b936-4bb5-a765-154c10ad5c61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:38.337299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1244d964-b936-4bb5-a765-154c10ad5c61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:12:38.338924 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1244d964-b936-4bb5-a765-154c10ad5c61 service nova] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:12:38.358749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 702/1416] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:12:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:12:39.261910 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b40146-f92f-4289-b506-d64fba58e679 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:39.262461 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b40146-f92f-4289-b506-d64fba58e679 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:39.262461 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b40146-f92f-4289-b506-d64fba58e679 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:39.263876 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 715/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 12:12:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:12:39.268803 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7f929-92cd-4030-89c7-2f77b0c03075 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:39.269257 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7f929-92cd-4030-89c7-2f77b0c03075 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:39.269357 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7f929-92cd-4030-89c7-2f77b0c03075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:39.345085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-8ad7f929-92cd-4030-89c7-2f77b0c03075 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:12:39.346292 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-8ad7f929-92cd-4030-89c7-2f77b0c03075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:39.373617 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:39.444405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 703/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 12:12:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:12:39.595044 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b40146-f92f-4289-b506-d64fba58e679 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:12:39.596608 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-16b40146-f92f-4289-b506-d64fba58e679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:39.739595 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b40146-f92f-4289-b506-d64fba58e679 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:12:39.780633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:39.856788 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8befae9d-ffb9-43c6-84e7-4ab7c83de153 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:12:39.857400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befae9d-ffb9-43c6-84e7-4ab7c83de153 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:39.857677 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befae9d-ffb9-43c6-84e7-4ab7c83de153 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:39.857946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befae9d-ffb9-43c6-84e7-4ab7c83de153 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:39.859590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8befae9d-ffb9-43c6-84e7-4ab7c83de153 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:39.953068 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8befae9d-ffb9-43c6-84e7-4ab7c83de153 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:12:39.954676 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8befae9d-ffb9-43c6-84e7-4ab7c83de153 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:12:39.972747 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 716/1419] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:12:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:12:40.867880 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24381650-743f-4971-9bb9-666f9f8acb8f None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:40.868516 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24381650-743f-4971-9bb9-666f9f8acb8f None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:40.868516 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24381650-743f-4971-9bb9-666f9f8acb8f None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:40.869825 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 704/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 12:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:12:40.874992 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf3be6-12b9-40cf-8ccd-fc786c57a5ed None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:40.875410 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf3be6-12b9-40cf-8ccd-fc786c57a5ed None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:40.875562 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf3be6-12b9-40cf-8ccd-fc786c57a5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:40.930623 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:40.953346 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-88bf3be6-12b9-40cf-8ccd-fc786c57a5ed None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:12:40.954558 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-88bf3be6-12b9-40cf-8ccd-fc786c57a5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:41.050468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 717/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 12:12:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:12:41.203664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24381650-743f-4971-9bb9-666f9f8acb8f None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:12:41.206485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-24381650-743f-4971-9bb9-666f9f8acb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:41.361900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24381650-743f-4971-9bb9-666f9f8acb8f None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:12:41.375079 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:41.486071 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2883096-c2a3-475a-88d4-d88c7d348a90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:12:41.486570 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2883096-c2a3-475a-88d4-d88c7d348a90 service nova] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:41.486843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2883096-c2a3-475a-88d4-d88c7d348a90 service nova] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:41.487110 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2883096-c2a3-475a-88d4-d88c7d348a90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:41.488767 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-a2883096-c2a3-475a-88d4-d88c7d348a90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:41.579899 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2883096-c2a3-475a-88d4-d88c7d348a90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:12:41.581509 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2883096-c2a3-475a-88d4-d88c7d348a90 service nova] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:12:41.600247 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 705/1422] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:12:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:12:42.461066 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09262f0f-bc05-4748-9581-e761d6ac3316 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:42.461652 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09262f0f-bc05-4748-9581-e761d6ac3316 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:42.461652 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09262f0f-bc05-4748-9581-e761d6ac3316 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:42.463503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 718/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 12:12:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:12:42.468532 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a087f-c1c3-401f-8c28-20d641f62d7f None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:42.468964 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a087f-c1c3-401f-8c28-20d641f62d7f None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:42.469084 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a087f-c1c3-401f-8c28-20d641f62d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:42.488964 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:42.548492 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-fd7a087f-c1c3-401f-8c28-20d641f62d7f None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:12:42.549709 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-fd7a087f-c1c3-401f-8c28-20d641f62d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:42.646274 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 706/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 12:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:12:42.797211 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09262f0f-bc05-4748-9581-e761d6ac3316 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:12:42.798739 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-09262f0f-bc05-4748-9581-e761d6ac3316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:42.908282 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:42.944760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09262f0f-bc05-4748-9581-e761d6ac3316 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:12:43.054135 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea9b59-73a1-4262-af97-ae00a81d091d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:12:43.054729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea9b59-73a1-4262-af97-ae00a81d091d service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:43.055003 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea9b59-73a1-4262-af97-ae00a81d091d service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:43.055266 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea9b59-73a1-4262-af97-ae00a81d091d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:43.057435 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b7ea9b59-73a1-4262-af97-ae00a81d091d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:43.132200 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:43.146365 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ea9b59-73a1-4262-af97-ae00a81d091d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:12:43.148011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ea9b59-73a1-4262-af97-ae00a81d091d service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:12:43.166759 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 719/1425] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:12:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:12:44.031322 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35481515-d8a3-49e1-bc2f-492e74c08040 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:44.031988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35481515-d8a3-49e1-bc2f-492e74c08040 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:44.031988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35481515-d8a3-49e1-bc2f-492e74c08040 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:44.033316 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 707/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 12:12:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:12:44.038879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f811435-d1dc-46f7-a1d1-62b059807458 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:44.039311 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f811435-d1dc-46f7-a1d1-62b059807458 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:44.039425 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f811435-d1dc-46f7-a1d1-62b059807458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:44.116664 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-3f811435-d1dc-46f7-a1d1-62b059807458 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:12:44.117869 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3f811435-d1dc-46f7-a1d1-62b059807458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:44.214611 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 720/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 12:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:12:44.381885 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35481515-d8a3-49e1-bc2f-492e74c08040 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:12:44.384620 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-35481515-d8a3-49e1-bc2f-492e74c08040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:44.469864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:44.533688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35481515-d8a3-49e1-bc2f-492e74c08040 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:12:44.647834 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a9669-c31c-40e8-8584-06378f84630d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:12:44.648356 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a9669-c31c-40e8-8584-06378f84630d service nova] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:44.648629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a9669-c31c-40e8-8584-06378f84630d service nova] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:44.648980 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a9669-c31c-40e8-8584-06378f84630d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:44.650572 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-711a9669-c31c-40e8-8584-06378f84630d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:44.698979 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:44.739987 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a9669-c31c-40e8-8584-06378f84630d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:12:44.741569 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a9669-c31c-40e8-8584-06378f84630d service nova] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:12:44.760371 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 708/1428] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:12:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:12:45.570187 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0729a3-5f98-42a3-9a7e-d52e5fb5f97c None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:45.571104 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0729a3-5f98-42a3-9a7e-d52e5fb5f97c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:45.571264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0729a3-5f98-42a3-9a7e-d52e5fb5f97c None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:45.573500 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 721/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 12:12:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:12:45.578904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac6721-861c-4ae0-a396-cee34d33c539 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:45.579373 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac6721-861c-4ae0-a396-cee34d33c539 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:45.579488 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac6721-861c-4ae0-a396-cee34d33c539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:45.656278 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-cfac6721-861c-4ae0-a396-cee34d33c539 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:12:45.657506 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-cfac6721-861c-4ae0-a396-cee34d33c539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:45.753300 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 709/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 12:12:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:12:45.908811 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0729a3-5f98-42a3-9a7e-d52e5fb5f97c None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:12:45.910243 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-bb0729a3-5f98-42a3-9a7e-d52e5fb5f97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:45.958669 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:46.054283 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0729a3-5f98-42a3-9a7e-d52e5fb5f97c None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:12:46.167257 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ec63a-9e39-49fb-ab7b-a2487a912be3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:12:46.167777 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ec63a-9e39-49fb-ab7b-a2487a912be3 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:46.168049 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ec63a-9e39-49fb-ab7b-a2487a912be3 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:46.168314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ec63a-9e39-49fb-ab7b-a2487a912be3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:46.169916 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a42ec63a-9e39-49fb-ab7b-a2487a912be3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:46.176874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:46.260948 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ec63a-9e39-49fb-ab7b-a2487a912be3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:12:46.262535 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ec63a-9e39-49fb-ab7b-a2487a912be3 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:12:46.283111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 722/1431] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:12:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:12:47.101306 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773851c-2c51-4183-ab0b-72377df194fe None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:47.101830 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773851c-2c51-4183-ab0b-72377df194fe None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:47.101830 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773851c-2c51-4183-ab0b-72377df194fe None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:47.103130 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 710/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 12:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:12:47.108150 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007f15e-dfe4-4c60-b66e-37a87c2d379b None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:47.108565 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007f15e-dfe4-4c60-b66e-37a87c2d379b None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:47.108683 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007f15e-dfe4-4c60-b66e-37a87c2d379b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:47.193425 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-6007f15e-dfe4-4c60-b66e-37a87c2d379b None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:12:47.194636 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6007f15e-dfe4-4c60-b66e-37a87c2d379b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:47.262066 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:47.290944 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 723/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 12:12:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:12:47.443117 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773851c-2c51-4183-ab0b-72377df194fe None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:12:47.444676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-0773851c-2c51-4183-ab0b-72377df194fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:47.462622 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:47.592345 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773851c-2c51-4183-ab0b-72377df194fe None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:12:47.680059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:47.707054 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09bcfc-b568-45c9-b6e9-9ddf387f1f90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:12:47.707686 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09bcfc-b568-45c9-b6e9-9ddf387f1f90 service nova] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:47.707968 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09bcfc-b568-45c9-b6e9-9ddf387f1f90 service nova] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:47.708241 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09bcfc-b568-45c9-b6e9-9ddf387f1f90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:47.709855 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-3b09bcfc-b568-45c9-b6e9-9ddf387f1f90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:47.800732 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09bcfc-b568-45c9-b6e9-9ddf387f1f90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:12:47.802366 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09bcfc-b568-45c9-b6e9-9ddf387f1f90 service nova] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:12:47.821825 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 711/1434] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:12:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:12:48.629687 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea9402-3b79-41b4-88f5-c62596e81515 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:48.630224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea9402-3b79-41b4-88f5-c62596e81515 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:48.630224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea9402-3b79-41b4-88f5-c62596e81515 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:48.631478 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 724/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 12:12:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:12:48.637028 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa654d6d-40e2-476f-805c-478a0c781690 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:48.637487 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa654d6d-40e2-476f-805c-478a0c781690 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:48.637585 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa654d6d-40e2-476f-805c-478a0c781690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:48.720773 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-aa654d6d-40e2-476f-805c-478a0c781690 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:12:48.721995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-aa654d6d-40e2-476f-805c-478a0c781690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:48.765014 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:48.824997 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 712/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 12:12:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:12:48.980404 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea9402-3b79-41b4-88f5-c62596e81515 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:12:48.981865 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-cbea9402-3b79-41b4-88f5-c62596e81515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:49.128779 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea9402-3b79-41b4-88f5-c62596e81515 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:12:49.185135 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:49.243622 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37844769-5729-4bc2-9d6f-ee05a8a18834 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:12:49.244146 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37844769-5729-4bc2-9d6f-ee05a8a18834 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:49.244429 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37844769-5729-4bc2-9d6f-ee05a8a18834 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:49.244706 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37844769-5729-4bc2-9d6f-ee05a8a18834 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:49.246334 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-37844769-5729-4bc2-9d6f-ee05a8a18834 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:49.335407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37844769-5729-4bc2-9d6f-ee05a8a18834 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:12:49.337557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37844769-5729-4bc2-9d6f-ee05a8a18834 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:12:49.358049 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 725/1437] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:12:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:12:50.169291 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b8f9f-d35f-44fb-8237-01804dfc750e None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:50.169919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b8f9f-d35f-44fb-8237-01804dfc750e None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:50.169919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b8f9f-d35f-44fb-8237-01804dfc750e None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:50.171173 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 713/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 12:12:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:12:50.176116 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e7026-376d-4e54-b4f3-7a5e6cad86a0 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:50.176563 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e7026-376d-4e54-b4f3-7a5e6cad86a0 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:50.176660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e7026-376d-4e54-b4f3-7a5e6cad86a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:50.253366 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-b22e7026-376d-4e54-b4f3-7a5e6cad86a0 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:12:50.254576 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b22e7026-376d-4e54-b4f3-7a5e6cad86a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:50.256676 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:50.351644 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 726/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 12:12:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:12:50.503014 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b8f9f-d35f-44fb-8237-01804dfc750e None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:12:50.504493 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-585b8f9f-d35f-44fb-8237-01804dfc750e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:50.652873 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b8f9f-d35f-44fb-8237-01804dfc750e None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:12:50.681456 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:50.776947 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b443d-079f-401a-ab6d-48a50145266a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:12:50.777470 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b443d-079f-401a-ab6d-48a50145266a service nova] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:50.777748 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b443d-079f-401a-ab6d-48a50145266a service nova] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:50.778019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b443d-079f-401a-ab6d-48a50145266a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:50.779941 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-a60b443d-079f-401a-ab6d-48a50145266a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:50.875643 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b443d-079f-401a-ab6d-48a50145266a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:12:50.877315 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b443d-079f-401a-ab6d-48a50145266a service nova] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:12:50.897373 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 714/1440] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:12:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:12:52.703380 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db0e42-90b5-4891-8513-0b11b0c2f5df None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:52.703947 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db0e42-90b5-4891-8513-0b11b0c2f5df None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:52.703947 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db0e42-90b5-4891-8513-0b11b0c2f5df None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:52.705218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 727/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 12:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:12:52.710379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0648dd-2d5e-4604-bff2-671f13daecf6 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:52.710789 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0648dd-2d5e-4604-bff2-671f13daecf6 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:52.710927 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0648dd-2d5e-4604-bff2-671f13daecf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:52.751662 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:52.780549 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-1c0648dd-2d5e-4604-bff2-671f13daecf6 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:12:52.781783 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-1c0648dd-2d5e-4604-bff2-671f13daecf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:52.896092 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 715/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 12:12:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:12:53.180941 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db0e42-90b5-4891-8513-0b11b0c2f5df None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:12:53.182524 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-89db0e42-90b5-4891-8513-0b11b0c2f5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:53.284827 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db0e42-90b5-4891-8513-0b11b0c2f5df None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:12:53.317259 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:53.359132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c032ebe-5e1f-40bf-8c65-7186cad27c5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:12:53.359753 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c032ebe-5e1f-40bf-8c65-7186cad27c5c service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:53.360026 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c032ebe-5e1f-40bf-8c65-7186cad27c5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:53.360291 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c032ebe-5e1f-40bf-8c65-7186cad27c5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:53.361877 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4c032ebe-5e1f-40bf-8c65-7186cad27c5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:53.468954 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c032ebe-5e1f-40bf-8c65-7186cad27c5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:12:53.470293 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c032ebe-5e1f-40bf-8c65-7186cad27c5c service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:12:53.490150 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 728/1443] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:12:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:12:55.583577 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df5791-edca-467b-96b1-e7e0aa322085 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:55.584198 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df5791-edca-467b-96b1-e7e0aa322085 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:55.584198 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df5791-edca-467b-96b1-e7e0aa322085 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:55.585441 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 716/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 12:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:12:55.591035 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90efc4-9b7b-4595-9480-cdf885a79979 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:55.591474 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90efc4-9b7b-4595-9480-cdf885a79979 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:55.591631 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90efc4-9b7b-4595-9480-cdf885a79979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:55.657210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-2d90efc4-9b7b-4595-9480-cdf885a79979 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:12:55.658435 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2d90efc4-9b7b-4595-9480-cdf885a79979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:55.671869 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:55.756129 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 729/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 12:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:12:56.038735 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df5791-edca-467b-96b1-e7e0aa322085 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:12:56.040349 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-b9df5791-edca-467b-96b1-e7e0aa322085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:56.144911 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df5791-edca-467b-96b1-e7e0aa322085 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:12:56.157642 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:56.226883 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0814a1f5-2399-4426-8453-f7513fb0a805 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:12:56.227482 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814a1f5-2399-4426-8453-f7513fb0a805 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:56.227793 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814a1f5-2399-4426-8453-f7513fb0a805 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:56.228063 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814a1f5-2399-4426-8453-f7513fb0a805 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:56.229681 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-0814a1f5-2399-4426-8453-f7513fb0a805 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:56.341469 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0814a1f5-2399-4426-8453-f7513fb0a805 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:12:56.342814 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0814a1f5-2399-4426-8453-f7513fb0a805 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:12:56.360540 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 717/1446] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:12:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:12:58.614740 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b75f8-5bf0-4725-830b-56179bb6ae3f None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:58.615304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b75f8-5bf0-4725-830b-56179bb6ae3f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:58.615304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b75f8-5bf0-4725-830b-56179bb6ae3f None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:58.616695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 730/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 12:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:12:58.622131 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9669f6-8985-4bb1-8485-82db599a6416 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:58.622584 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9669f6-8985-4bb1-8485-82db599a6416 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:58.622711 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9669f6-8985-4bb1-8485-82db599a6416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:58.672684 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:58.691174 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-3f9669f6-8985-4bb1-8485-82db599a6416 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:12:58.692441 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-3f9669f6-8985-4bb1-8485-82db599a6416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:58.792169 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 718/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 12:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:12:59.081627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b75f8-5bf0-4725-830b-56179bb6ae3f None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:12:59.083195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-eb2b75f8-5bf0-4725-830b-56179bb6ae3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:59.187515 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b75f8-5bf0-4725-830b-56179bb6ae3f None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:12:59.228074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:59.264768 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37945f29-5369-4c1e-85b1-4e2cfac824a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:12:59.265366 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37945f29-5369-4c1e-85b1-4e2cfac824a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:12:59.265640 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37945f29-5369-4c1e-85b1-4e2cfac824a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:12:59.265907 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37945f29-5369-4c1e-85b1-4e2cfac824a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:12:59.267490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-37945f29-5369-4c1e-85b1-4e2cfac824a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:12:59.378246 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37945f29-5369-4c1e-85b1-4e2cfac824a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:12:59.379735 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37945f29-5369-4c1e-85b1-4e2cfac824a1 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:12:59.399042 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 731/1449] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:12:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:13:01.560989 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86a815-fd31-4cdf-b282-0baea8fb837b None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:01.561577 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86a815-fd31-4cdf-b282-0baea8fb837b None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:01.561577 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86a815-fd31-4cdf-b282-0baea8fb837b None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:01.562855 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 719/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 12:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:13:01.568313 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f87094-01d8-4c14-bcf4-234e2c5ea3d1 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:01.568793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f87094-01d8-4c14-bcf4-234e2c5ea3d1 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:01.568900 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f87094-01d8-4c14-bcf4-234e2c5ea3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:01.638809 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-62f87094-01d8-4c14-bcf4-234e2c5ea3d1 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:13:01.640059 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-62f87094-01d8-4c14-bcf4-234e2c5ea3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:01.662128 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:01.739039 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 732/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 12:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:13:02.021753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86a815-fd31-4cdf-b282-0baea8fb837b None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:02.023369 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-4f86a815-fd31-4cdf-b282-0baea8fb837b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:02.129299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86a815-fd31-4cdf-b282-0baea8fb837b None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:02.149903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:02.207478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beda230-ee6b-4b68-b73d-1e48f1e54797 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:02.208111 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beda230-ee6b-4b68-b73d-1e48f1e54797 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:02.208384 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beda230-ee6b-4b68-b73d-1e48f1e54797 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:02.208652 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beda230-ee6b-4b68-b73d-1e48f1e54797 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:02.210297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-0beda230-ee6b-4b68-b73d-1e48f1e54797 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:02.322322 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beda230-ee6b-4b68-b73d-1e48f1e54797 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:02.323789 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beda230-ee6b-4b68-b73d-1e48f1e54797 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:02.342267 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 720/1452] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:13:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:13:04.325823 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c2015-b337-4209-91a7-7d6d2fbba7f8 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:04.326411 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c2015-b337-4209-91a7-7d6d2fbba7f8 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:04.326411 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c2015-b337-4209-91a7-7d6d2fbba7f8 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:04.327782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 733/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 12:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:13:04.333278 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139506b6-42b3-4e30-88f0-fff872765228 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:04.333738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139506b6-42b3-4e30-88f0-fff872765228 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:04.333873 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139506b6-42b3-4e30-88f0-fff872765228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:04.400719 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:04.403826 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-139506b6-42b3-4e30-88f0-fff872765228 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:13:04.405890 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-139506b6-42b3-4e30-88f0-fff872765228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:04.509080 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 721/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 12:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:13:04.790697 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c2015-b337-4209-91a7-7d6d2fbba7f8 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:04.792335 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a14c2015-b337-4209-91a7-7d6d2fbba7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:04.899006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c2015-b337-4209-91a7-7d6d2fbba7f8 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:04.948501 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:04.973810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6105336-7ddd-4e8f-a9f0-90befb8fee32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:04.974534 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6105336-7ddd-4e8f-a9f0-90befb8fee32 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:04.974749 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6105336-7ddd-4e8f-a9f0-90befb8fee32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:04.975023 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6105336-7ddd-4e8f-a9f0-90befb8fee32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:04.976699 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d6105336-7ddd-4e8f-a9f0-90befb8fee32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:05.087631 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6105336-7ddd-4e8f-a9f0-90befb8fee32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:05.089776 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6105336-7ddd-4e8f-a9f0-90befb8fee32 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:05.110192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 734/1455] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:13:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:13:07.080381 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2034d70-540e-4747-8812-18b5733b38c8 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:07.080956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2034d70-540e-4747-8812-18b5733b38c8 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:07.080956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2034d70-540e-4747-8812-18b5733b38c8 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:07.082237 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 722/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 12:13:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:13:07.087926 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b8b7d-0a6c-4c68-9677-d7b264d22f98 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:07.088396 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b8b7d-0a6c-4c68-9677-d7b264d22f98 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:07.088504 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b8b7d-0a6c-4c68-9677-d7b264d22f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:07.156753 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-d32b8b7d-0a6c-4c68-9677-d7b264d22f98 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:13:07.157897 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d32b8b7d-0a6c-4c68-9677-d7b264d22f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:07.190288 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:07.261459 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 735/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 12:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:13:07.545259 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2034d70-540e-4747-8812-18b5733b38c8 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:07.546782 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-f2034d70-540e-4747-8812-18b5733b38c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:07.653760 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2034d70-540e-4747-8812-18b5733b38c8 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:07.682969 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:07.729269 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe717e20-cb05-4dd7-9a92-95f6b323053b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:07.729874 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe717e20-cb05-4dd7-9a92-95f6b323053b service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:07.730156 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe717e20-cb05-4dd7-9a92-95f6b323053b service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:07.730432 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe717e20-cb05-4dd7-9a92-95f6b323053b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:07.732274 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-fe717e20-cb05-4dd7-9a92-95f6b323053b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:07.843140 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe717e20-cb05-4dd7-9a92-95f6b323053b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:07.844638 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe717e20-cb05-4dd7-9a92-95f6b323053b service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:07.862515 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 723/1458] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:13:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:13:09.905780 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172eada4-8d7b-43e2-99f5-14782be908b4 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:09.906383 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172eada4-8d7b-43e2-99f5-14782be908b4 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:09.906383 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172eada4-8d7b-43e2-99f5-14782be908b4 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:09.907707 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 736/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 12:13:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:13:09.914648 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd264f-f6f6-4253-b47b-06576490ea67 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:09.915315 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd264f-f6f6-4253-b47b-06576490ea67 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:09.915315 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd264f-f6f6-4253-b47b-06576490ea67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:09.984550 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-98bd264f-f6f6-4253-b47b-06576490ea67 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:13:09.985768 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-98bd264f-f6f6-4253-b47b-06576490ea67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:09.994925 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:10.203263 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 724/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 12:13:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:13:10.485212 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172eada4-8d7b-43e2-99f5-14782be908b4 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:10.486788 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-172eada4-8d7b-43e2-99f5-14782be908b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:10.591415 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172eada4-8d7b-43e2-99f5-14782be908b4 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:10.647085 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:10.667313 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40131aed-19af-4919-9368-cb3628cd8653 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:10.667957 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40131aed-19af-4919-9368-cb3628cd8653 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:10.668234 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40131aed-19af-4919-9368-cb3628cd8653 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:10.668504 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40131aed-19af-4919-9368-cb3628cd8653 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:10.670139 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-40131aed-19af-4919-9368-cb3628cd8653 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:10.781430 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40131aed-19af-4919-9368-cb3628cd8653 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:10.782878 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40131aed-19af-4919-9368-cb3628cd8653 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:10.805230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 737/1461] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:13:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:13:12.764267 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab2d66-0ff0-4ee0-a7ab-59a67b756198 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:12.764780 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab2d66-0ff0-4ee0-a7ab-59a67b756198 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:12.764780 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab2d66-0ff0-4ee0-a7ab-59a67b756198 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:12.766053 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 725/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 12:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:13:12.771796 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba2c6d-f7b7-45c1-8729-37fff55f491c None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:12.772313 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba2c6d-f7b7-45c1-8729-37fff55f491c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:12.772407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba2c6d-f7b7-45c1-8729-37fff55f491c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:12.841025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-b8ba2c6d-f7b7-45c1-8729-37fff55f491c None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:13:12.842248 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b8ba2c6d-f7b7-45c1-8729-37fff55f491c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:12.881540 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:12.943481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 738/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 12:13:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:13:13.232744 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab2d66-0ff0-4ee0-a7ab-59a67b756198 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:13.234278 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-5bab2d66-0ff0-4ee0-a7ab-59a67b756198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:13.340289 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab2d66-0ff0-4ee0-a7ab-59a67b756198 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:13.378013 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:13.415400 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6862ca-6698-471d-bcf9-6f4b64aab94f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:13.415947 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6862ca-6698-471d-bcf9-6f4b64aab94f service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:13.416231 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6862ca-6698-471d-bcf9-6f4b64aab94f service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:13.416498 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6862ca-6698-471d-bcf9-6f4b64aab94f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:13.418115 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-ac6862ca-6698-471d-bcf9-6f4b64aab94f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:13.528244 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6862ca-6698-471d-bcf9-6f4b64aab94f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:13.529923 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6862ca-6698-471d-bcf9-6f4b64aab94f service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:13.547644 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 726/1464] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:13:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:13:15.479488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42b567-9181-47e7-850a-60be3fae697f None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:15.480030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42b567-9181-47e7-850a-60be3fae697f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:15.480030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42b567-9181-47e7-850a-60be3fae697f None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:15.481337 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 739/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 12:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:13:15.486686 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178085c3-78cd-41dd-a093-44f86fb418d6 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:15.487106 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178085c3-78cd-41dd-a093-44f86fb418d6 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:15.487211 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178085c3-78cd-41dd-a093-44f86fb418d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:15.552895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-178085c3-78cd-41dd-a093-44f86fb418d6 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:13:15.554665 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-178085c3-78cd-41dd-a093-44f86fb418d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:15.573393 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:15.652889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 727/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 12:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:13:15.937457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42b567-9181-47e7-850a-60be3fae697f None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:15.939126 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1c42b567-9181-47e7-850a-60be3fae697f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:16.041682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42b567-9181-47e7-850a-60be3fae697f None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:16.107739 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:16.114746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c6184-242e-447d-8b31-61c6d150e784 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:16.115362 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c6184-242e-447d-8b31-61c6d150e784 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:16.115681 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c6184-242e-447d-8b31-61c6d150e784 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:16.115954 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c6184-242e-447d-8b31-61c6d150e784 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:16.118189 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c23c6184-242e-447d-8b31-61c6d150e784 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:16.227892 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23c6184-242e-447d-8b31-61c6d150e784 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:16.229376 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23c6184-242e-447d-8b31-61c6d150e784 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:16.248671 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 740/1467] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:13:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:13:18.203038 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd15d76-5315-4242-8efa-57665866a302 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:18.203575 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd15d76-5315-4242-8efa-57665866a302 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:18.203575 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd15d76-5315-4242-8efa-57665866a302 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:18.204868 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 728/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 12:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:13:18.210627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112dc335-ddee-4ab4-bda6-55c3902bcea9 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:18.211095 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112dc335-ddee-4ab4-bda6-55c3902bcea9 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:18.211208 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112dc335-ddee-4ab4-bda6-55c3902bcea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:18.277659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-112dc335-ddee-4ab4-bda6-55c3902bcea9 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:13:18.279119 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-112dc335-ddee-4ab4-bda6-55c3902bcea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:18.329567 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:18.381057 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 741/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 12:13:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:13:18.671898 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd15d76-5315-4242-8efa-57665866a302 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:18.673492 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-0dd15d76-5315-4242-8efa-57665866a302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:18.778817 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd15d76-5315-4242-8efa-57665866a302 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:18.828178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:18.855877 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35abbd25-263e-4bea-8704-701920cdbd90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:18.856485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abbd25-263e-4bea-8704-701920cdbd90 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:18.856762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abbd25-263e-4bea-8704-701920cdbd90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:18.857038 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abbd25-263e-4bea-8704-701920cdbd90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:18.858652 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-35abbd25-263e-4bea-8704-701920cdbd90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:18.968186 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35abbd25-263e-4bea-8704-701920cdbd90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:18.969517 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35abbd25-263e-4bea-8704-701920cdbd90 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:18.986961 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 729/1470] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:13:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:13:28.379176 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=22 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:28.984744 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=66 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:38.768763 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba25ef-fd69-4624-839f-0acbb3da0aad None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:38.769350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba25ef-fd69-4624-839f-0acbb3da0aad None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:38.769350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba25ef-fd69-4624-839f-0acbb3da0aad None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:38.770585 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 742/1471] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:13:38.776515 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707935c-578c-427c-9e78-2171c4abe508 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:38.776981 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707935c-578c-427c-9e78-2171c4abe508 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:38.777104 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707935c-578c-427c-9e78-2171c4abe508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:38.843713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-6707935c-578c-427c-9e78-2171c4abe508 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:13:38.844931 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-6707935c-578c-427c-9e78-2171c4abe508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:38.944427 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 730/1472] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:13:39.253528 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce51aaf-1d28-444b-b01b-55288c08784a None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:39.253816 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce51aaf-1d28-444b-b01b-55288c08784a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:39.254094 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce51aaf-1d28-444b-b01b-55288c08784a None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:39.255429 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 743/1473] 10.222.0.16 () {54 vars in 961 bytes} [Mon Sep 5 12:13:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:13:39.260707 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f67c0-2607-4804-8808-71e2222f0c02 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:39.260987 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f67c0-2607-4804-8808-71e2222f0c02 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:39.261267 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f67c0-2607-4804-8808-71e2222f0c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:39.337034 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-636f67c0-2607-4804-8808-71e2222f0c02 None None] MFA Rules not processed for user `ed75c0be809d4a0eb0e2ccc59805b1ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:13:39.338215 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-636f67c0-2607-4804-8808-71e2222f0c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:39.359155 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:39.419027 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 731/1474] 10.222.0.16 () {58 vars in 1058 bytes} [Mon Sep 5 12:13:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:13:39.426645 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce51aaf-1d28-444b-b01b-55288c08784a None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:39.428207 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7ce51aaf-1d28-444b-b01b-55288c08784a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:39.533485 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce51aaf-1d28-444b-b01b-55288c08784a None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:39.605904 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95449d7-7ec0-4615-8a1c-0ba1d37b7c86 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed75c0be809d4a0eb0e2ccc59805b1ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:39.606504 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95449d7-7ec0-4615-8a1c-0ba1d37b7c86 service glance] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:39.606775 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95449d7-7ec0-4615-8a1c-0ba1d37b7c86 service glance] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:39.607040 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95449d7-7ec0-4615-8a1c-0ba1d37b7c86 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:39.608651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f95449d7-7ec0-4615-8a1c-0ba1d37b7c86 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:39.661097 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:39.715345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95449d7-7ec0-4615-8a1c-0ba1d37b7c86 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:39.716711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95449d7-7ec0-4615-8a1c-0ba1d37b7c86 service glance] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:39.734624 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 744/1475] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:13:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:13:41.749322 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619809b8-4b32-4f2e-8933-ab8b83e9a608 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:41.749917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619809b8-4b32-4f2e-8933-ab8b83e9a608 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:41.749917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619809b8-4b32-4f2e-8933-ab8b83e9a608 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:41.751209 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 732/1476] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:13:41.756300 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ac5f2-fa14-46cf-b93f-11f8d3ede01f None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:41.756732 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ac5f2-fa14-46cf-b93f-11f8d3ede01f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:41.756847 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ac5f2-fa14-46cf-b93f-11f8d3ede01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:41.824317 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-4f4ac5f2-fa14-46cf-b93f-11f8d3ede01f None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:13:41.825529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4f4ac5f2-fa14-46cf-b93f-11f8d3ede01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:41.922157 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 745/1477] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:13:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:13:42.191161 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619809b8-4b32-4f2e-8933-ab8b83e9a608 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:42.192735 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-619809b8-4b32-4f2e-8933-ab8b83e9a608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:42.317370 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619809b8-4b32-4f2e-8933-ab8b83e9a608 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:42.366521 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:42.418865 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb2388-2b62-4939-a1c4-9c1998ca3a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:42.419463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb2388-2b62-4939-a1c4-9c1998ca3a68 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:42.419768 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb2388-2b62-4939-a1c4-9c1998ca3a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:42.420038 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb2388-2b62-4939-a1c4-9c1998ca3a68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:42.420673 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb2388-2b62-4939-a1c4-9c1998ca3a68 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:42.422417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: WARNING py.warnings [None req-f3fb2388-2b62-4939-a1c4-9c1998ca3a68 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 12:13:42.422417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: warnings.warn(msg) Sep 05 12:13:42.422417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: Sep 05 12:13:42.423560 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb2388-2b62-4939-a1c4-9c1998ca3a68 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:42.442267 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 733/1478] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Sep 5 12:13:42 2022] GET /identity/v3/projects => generated 2553 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:13:44.400621 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76200b3a-dc1f-4227-bf99-6b29b97ab044 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:44.401156 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76200b3a-dc1f-4227-bf99-6b29b97ab044 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:44.401156 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76200b3a-dc1f-4227-bf99-6b29b97ab044 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:44.402556 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 746/1479] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:13:44.408116 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964536e-0cd4-4e87-b58d-1de03cc6312a None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:44.408570 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964536e-0cd4-4e87-b58d-1de03cc6312a None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:44.408680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964536e-0cd4-4e87-b58d-1de03cc6312a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:44.473829 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-9964536e-0cd4-4e87-b58d-1de03cc6312a None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:13:44.475029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-9964536e-0cd4-4e87-b58d-1de03cc6312a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:44.572930 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 734/1480] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:13:44.868027 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76200b3a-dc1f-4227-bf99-6b29b97ab044 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:44.869580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-76200b3a-dc1f-4227-bf99-6b29b97ab044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:44.879307 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:45.016630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76200b3a-dc1f-4227-bf99-6b29b97ab044 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:45.096388 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:45.130415 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60089b59-1341-4c7b-a2bf-34e788097a73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:45.131451 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60089b59-1341-4c7b-a2bf-34e788097a73 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:45.131827 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60089b59-1341-4c7b-a2bf-34e788097a73 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:45.132101 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60089b59-1341-4c7b-a2bf-34e788097a73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:45.133903 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-60089b59-1341-4c7b-a2bf-34e788097a73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:45.222431 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60089b59-1341-4c7b-a2bf-34e788097a73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:45.224066 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60089b59-1341-4c7b-a2bf-34e788097a73 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:45.241898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 747/1481] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:13:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:13:47.368247 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cafc59a-2be3-4904-9e62-a936f79bd2dc None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:47.369052 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cafc59a-2be3-4904-9e62-a936f79bd2dc None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:47.369217 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cafc59a-2be3-4904-9e62-a936f79bd2dc None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:47.370829 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 735/1482] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:13:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:13:47.378149 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9088a2f-549c-43f7-880b-d99173424bb2 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:47.378558 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9088a2f-549c-43f7-880b-d99173424bb2 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:47.378696 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9088a2f-549c-43f7-880b-d99173424bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:47.444538 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-f9088a2f-549c-43f7-880b-d99173424bb2 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:13:47.445669 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f9088a2f-549c-43f7-880b-d99173424bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:47.500969 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:47.550289 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 748/1483] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:13:47.836278 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cafc59a-2be3-4904-9e62-a936f79bd2dc None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:47.837807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-3cafc59a-2be3-4904-9e62-a936f79bd2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:47.862206 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:47.944375 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cafc59a-2be3-4904-9e62-a936f79bd2dc None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:48.016346 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5941f9a-8e77-403a-bdb1-3bd46135ea70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:48.016943 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5941f9a-8e77-403a-bdb1-3bd46135ea70 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:48.017217 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5941f9a-8e77-403a-bdb1-3bd46135ea70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:48.017486 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5941f9a-8e77-403a-bdb1-3bd46135ea70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:48.019085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-c5941f9a-8e77-403a-bdb1-3bd46135ea70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:48.095155 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:48.130135 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5941f9a-8e77-403a-bdb1-3bd46135ea70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:48.132126 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5941f9a-8e77-403a-bdb1-3bd46135ea70 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:48.150275 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 736/1484] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:13:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:13:50.125488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a6a57-1131-490e-ad4e-99939708e252 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:50.126053 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a6a57-1131-490e-ad4e-99939708e252 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:50.126053 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a6a57-1131-490e-ad4e-99939708e252 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:50.127369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 749/1485] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:13:50.133234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54bcce-a2bd-4698-b686-670f765b6769 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:50.133713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54bcce-a2bd-4698-b686-670f765b6769 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:50.133821 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54bcce-a2bd-4698-b686-670f765b6769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:50.200719 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-db54bcce-a2bd-4698-b686-670f765b6769 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:13:50.201924 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-db54bcce-a2bd-4698-b686-670f765b6769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:50.296871 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 737/1486] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:13:50.585978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a6a57-1131-490e-ad4e-99939708e252 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:50.587542 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-0f8a6a57-1131-490e-ad4e-99939708e252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:50.689513 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a6a57-1131-490e-ad4e-99939708e252 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:50.759136 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:50.761088 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87dc14-15c3-4635-bc36-83bbadbed14f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:50.761734 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87dc14-15c3-4635-bc36-83bbadbed14f service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:50.762126 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87dc14-15c3-4635-bc36-83bbadbed14f service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:50.762577 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87dc14-15c3-4635-bc36-83bbadbed14f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:50.764600 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-0e87dc14-15c3-4635-bc36-83bbadbed14f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:50.876199 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87dc14-15c3-4635-bc36-83bbadbed14f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:50.877543 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87dc14-15c3-4635-bc36-83bbadbed14f service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:50.895688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 750/1487] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:13:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:13:52.892756 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227644e5-c50f-44e0-be49-d8aebba323af None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:52.893320 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227644e5-c50f-44e0-be49-d8aebba323af None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:52.893320 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227644e5-c50f-44e0-be49-d8aebba323af None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:52.894705 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 738/1488] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:13:52.900643 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01727461-cf64-4c11-8bd5-90df144a2370 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:52.901075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01727461-cf64-4c11-8bd5-90df144a2370 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:52.901177 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01727461-cf64-4c11-8bd5-90df144a2370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:52.966899 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-01727461-cf64-4c11-8bd5-90df144a2370 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:13:52.968130 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-01727461-cf64-4c11-8bd5-90df144a2370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:53.020054 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:53.064956 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 751/1489] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:13:53.365559 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227644e5-c50f-44e0-be49-d8aebba323af None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:53.367253 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-227644e5-c50f-44e0-be49-d8aebba323af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:53.404325 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:53.472425 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227644e5-c50f-44e0-be49-d8aebba323af None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:53.546544 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8e57e-4f30-4a80-848d-70f677b07035 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:53.547139 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8e57e-4f30-4a80-848d-70f677b07035 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:53.547411 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8e57e-4f30-4a80-848d-70f677b07035 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:53.547712 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8e57e-4f30-4a80-848d-70f677b07035 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:53.549319 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-b9c8e57e-4f30-4a80-848d-70f677b07035 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:53.628575 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:53.663857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8e57e-4f30-4a80-848d-70f677b07035 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:53.666151 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8e57e-4f30-4a80-848d-70f677b07035 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:53.684861 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 739/1490] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:13:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:13:55.832516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a62d4-6b9e-4ebb-8b11-60aa1d7cf771 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:55.833135 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a62d4-6b9e-4ebb-8b11-60aa1d7cf771 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:55.833135 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a62d4-6b9e-4ebb-8b11-60aa1d7cf771 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:55.834412 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 752/1491] 10.222.0.16 () {54 vars in 927 bytes} [Mon Sep 5 12:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:13:55.839757 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefaa1b-27f2-4cd3-805b-ac1bcaa2e7ef None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:55.840254 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefaa1b-27f2-4cd3-805b-ac1bcaa2e7ef None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:55.840356 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefaa1b-27f2-4cd3-805b-ac1bcaa2e7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:55.908486 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-eaefaa1b-27f2-4cd3-805b-ac1bcaa2e7ef None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:13:55.909710 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-eaefaa1b-27f2-4cd3-805b-ac1bcaa2e7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:56.005626 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 740/1492] 10.222.0.16 () {58 vars in 1024 bytes} [Mon Sep 5 12:13:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:13:56.295415 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a62d4-6b9e-4ebb-8b11-60aa1d7cf771 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:56.297010 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ee8a62d4-6b9e-4ebb-8b11-60aa1d7cf771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:56.401456 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a62d4-6b9e-4ebb-8b11-60aa1d7cf771 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:56.474610 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22512367-80c0-4e0e-bf6b-d625d11737cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:56.475218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22512367-80c0-4e0e-bf6b-d625d11737cc service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:56.475496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22512367-80c0-4e0e-bf6b-d625d11737cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:56.475796 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22512367-80c0-4e0e-bf6b-d625d11737cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:56.477389 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-22512367-80c0-4e0e-bf6b-d625d11737cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:56.486357 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:56.585929 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22512367-80c0-4e0e-bf6b-d625d11737cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:56.587248 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22512367-80c0-4e0e-bf6b-d625d11737cc service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:56.604669 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 753/1493] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:13:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:14:06.003852 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=82 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:14:06.602472 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=50 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:15:09.721834 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436aaf0-fb1e-44f4-955d-a219fefb0c9f None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:15:09.722994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436aaf0-fb1e-44f4-955d-a219fefb0c9f None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:15:09.722994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436aaf0-fb1e-44f4-955d-a219fefb0c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:15:09.806335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-9436aaf0-fb1e-44f4-955d-a219fefb0c9f None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:15:09.807605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-9436aaf0-fb1e-44f4-955d-a219fefb0c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:15:09.903874 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 741/1494] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:15:09.911872 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd673f-3d4c-4bb3-bba3-f4bc4deae403 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:15:09.912369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd673f-3d4c-4bb3-bba3-f4bc4deae403 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:15:09.912474 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd673f-3d4c-4bb3-bba3-f4bc4deae403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:15:09.987591 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-42fd673f-3d4c-4bb3-bba3-f4bc4deae403 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:15:09.988767 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-42fd673f-3d4c-4bb3-bba3-f4bc4deae403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:15:10.084943 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 754/1495] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:15:10.089626 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436aaf0-fb1e-44f4-955d-a219fefb0c9f None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:15:10.090807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-9436aaf0-fb1e-44f4-955d-a219fefb0c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:15:10.102079 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436aaf0-fb1e-44f4-955d-a219fefb0c9f None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:15:10.162577 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:15:10.196782 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9ae09-1c8d-4e69-a969-93696f6d6736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:15:10.197388 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9ae09-1c8d-4e69-a969-93696f6d6736 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:15:10.197660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9ae09-1c8d-4e69-a969-93696f6d6736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:15:10.197925 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9ae09-1c8d-4e69-a969-93696f6d6736 admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:15:10.198561 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e9ae09-1c8d-4e69-a969-93696f6d6736 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:15:10.198857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e9ae09-1c8d-4e69-a969-93696f6d6736 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:15:10.199929 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: WARNING py.warnings [None req-89e9ae09-1c8d-4e69-a969-93696f6d6736 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 05 12:15:10.199929 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: warnings.warn(msg) Sep 05 12:15:10.199929 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: Sep 05 12:15:10.200798 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e9ae09-1c8d-4e69-a969-93696f6d6736 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:15:10.212586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 742/1496] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:15:10.221328 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd673f-3d4c-4bb3-bba3-f4bc4deae403 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:15:10.230340 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd673f-3d4c-4bb3-bba3-f4bc4deae403 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:15:10.290355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:15:10.325497 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc559c-d272-4469-9e45-759514a7ffb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:15:10.326099 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc559c-d272-4469-9e45-759514a7ffb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:15:10.326371 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc559c-d272-4469-9e45-759514a7ffb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:15:10.326637 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc559c-d272-4469-9e45-759514a7ffb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:15:10.327278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc559c-d272-4469-9e45-759514a7ffb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:15:10.329048 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: WARNING py.warnings [None req-7cdc559c-d272-4469-9e45-759514a7ffb2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 12:15:10.329048 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: warnings.warn(msg) Sep 05 12:15:10.329048 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: Sep 05 12:15:10.330242 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc559c-d272-4469-9e45-759514a7ffb2 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:15:10.387269 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 755/1497] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:15:10 2022] POST /identity/v3/projects => generated 375 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:15:10.391642 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9ae09-1c8d-4e69-a969-93696f6d6736 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:15:10.393192 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-89e9ae09-1c8d-4e69-a969-93696f6d6736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:15:10.427412 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9ae09-1c8d-4e69-a969-93696f6d6736 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:15:10.521332 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf259c2-cb7d-4676-81af-e4b5dc927160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:15:10.521931 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf259c2-cb7d-4676-81af-e4b5dc927160 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:15:10.522208 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf259c2-cb7d-4676-81af-e4b5dc927160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:15:10.522481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf259c2-cb7d-4676-81af-e4b5dc927160 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:15:10.523116 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf259c2-cb7d-4676-81af-e4b5dc927160 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:15:10.524154 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: WARNING py.warnings [None req-0cf259c2-cb7d-4676-81af-e4b5dc927160 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 12:15:10.524154 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: warnings.warn(msg) Sep 05 12:15:10.524154 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: Sep 05 12:15:10.525360 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf259c2-cb7d-4676-81af-e4b5dc927160 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:15:10.569481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 743/1498] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:15:10 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:15:10.573716 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc559c-d272-4469-9e45-759514a7ffb2 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:15:10.575281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7cdc559c-d272-4469-9e45-759514a7ffb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:15:10.636848 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc559c-d272-4469-9e45-759514a7ffb2 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:15:10.719786 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:15:10.731674 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb54864-6b44-4698-93df-020bb36c4b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:15:10.732282 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb54864-6b44-4698-93df-020bb36c4b34 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:15:10.732562 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb54864-6b44-4698-93df-020bb36c4b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:15:10.732835 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb54864-6b44-4698-93df-020bb36c4b34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:15:10.733468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb54864-6b44-4698-93df-020bb36c4b34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:15:10.734476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: WARNING py.warnings [None req-5fb54864-6b44-4698-93df-020bb36c4b34 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 05 12:15:10.734476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: warnings.warn(msg) Sep 05 12:15:10.734476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: Sep 05 12:15:10.735360 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb54864-6b44-4698-93df-020bb36c4b34 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:15:10.748407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 756/1499] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:15:10.752490 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf259c2-cb7d-4676-81af-e4b5dc927160 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:15:10.754005 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-0cf259c2-cb7d-4676-81af-e4b5dc927160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:15:10.765342 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf259c2-cb7d-4676-81af-e4b5dc927160 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:15:10.791674 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:15:10.867096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ed7d6-52bd-4ca6-917d-3c1f10ae286e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:15:10.867658 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ed7d6-52bd-4ca6-917d-3c1f10ae286e admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:15:10.867961 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ed7d6-52bd-4ca6-917d-3c1f10ae286e admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:15:10.868228 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ed7d6-52bd-4ca6-917d-3c1f10ae286e admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:15:10.868900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ed7d6-52bd-4ca6-917d-3c1f10ae286e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:15:10.869856 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: WARNING py.warnings [None req-4c6ed7d6-52bd-4ca6-917d-3c1f10ae286e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 05 12:15:10.869856 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: warnings.warn(msg) Sep 05 12:15:10.869856 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: Sep 05 12:15:10.870741 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ed7d6-52bd-4ca6-917d-3c1f10ae286e admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:15:10.883777 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 744/1500] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:15:10.888024 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb54864-6b44-4698-93df-020bb36c4b34 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:15:10.889582 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5fb54864-6b44-4698-93df-020bb36c4b34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:15:10.922946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb54864-6b44-4698-93df-020bb36c4b34 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:15:11.015446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff843b2b-d670-4059-b0a7-56db435e4a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:15:11.016126 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff843b2b-d670-4059-b0a7-56db435e4a63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:15:11.016399 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff843b2b-d670-4059-b0a7-56db435e4a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:15:11.016667 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff843b2b-d670-4059-b0a7-56db435e4a63 admin admin] PATH_INFO: `/v3/projects/c36b205827634232ac0b6b89e597c33d/users/dc2636095ab84f23b58023da9b207270/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:15:11.039528 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff843b2b-d670-4059-b0a7-56db435e4a63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c36b205827634232ac0b6b89e597c33d, user_id=dc2636095ab84f23b58023da9b207270, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:15:11.040574 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: WARNING py.warnings [None req-ff843b2b-d670-4059-b0a7-56db435e4a63 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 12:15:11.040574 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: warnings.warn(msg) Sep 05 12:15:11.040574 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: Sep 05 12:15:11.042545 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff843b2b-d670-4059-b0a7-56db435e4a63 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:15:11.066715 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 757/1501] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:15:10 2022] PUT /identity/v3/projects/c36b205827634232ac0b6b89e597c33d/users/dc2636095ab84f23b58023da9b207270/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:15:11.079118 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405fb02-5c7c-479e-aaa7-e47a7d8c4ec3 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:15:11.079410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405fb02-5c7c-479e-aaa7-e47a7d8c4ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:15:11.079722 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405fb02-5c7c-479e-aaa7-e47a7d8c4ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:15:11.139555 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-8405fb02-5c7c-479e-aaa7-e47a7d8c4ec3 None None] MFA Rules not processed for user `dc2636095ab84f23b58023da9b207270`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:15:11.140690 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-8405fb02-5c7c-479e-aaa7-e47a7d8c4ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:15:11.220127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 745/1502] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:15:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:15:11.235051 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a27c6-212f-44e8-993e-68674c0a2b96 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:15:11.235339 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a27c6-212f-44e8-993e-68674c0a2b96 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:15:11.235651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a27c6-212f-44e8-993e-68674c0a2b96 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:15:11.237001 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 758/1503] 10.222.0.16 () {54 vars in 964 bytes} [Mon Sep 5 12:15:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:15:11.242191 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfab694-0c72-4742-a358-5ffc8fd9d487 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:15:11.242484 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfab694-0c72-4742-a358-5ffc8fd9d487 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:15:11.242759 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfab694-0c72-4742-a358-5ffc8fd9d487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:15:11.276296 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:15:11.323724 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-9cfab694-0c72-4742-a358-5ffc8fd9d487 None None] MFA Rules not processed for user `e8a47a0d9bfd473cbf0147d7801442d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:15:11.324975 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-9cfab694-0c72-4742-a358-5ffc8fd9d487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:15:11.404698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 746/1504] 10.222.0.16 () {58 vars in 1061 bytes} [Mon Sep 5 12:15:11 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:15:11.412615 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a27c6-212f-44e8-993e-68674c0a2b96 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:15:11.414192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a38a27c6-212f-44e8-993e-68674c0a2b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:15:11.465232 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:15:11.520973 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a27c6-212f-44e8-993e-68674c0a2b96 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:15:11.597821 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c04214-8ede-48e3-9165-28bb1f7adf0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:15:11.598425 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c04214-8ede-48e3-9165-28bb1f7adf0b service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:15:11.598698 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c04214-8ede-48e3-9165-28bb1f7adf0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:15:11.598962 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c04214-8ede-48e3-9165-28bb1f7adf0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:15:11.600600 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-10c04214-8ede-48e3-9165-28bb1f7adf0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:15:11.700514 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c04214-8ede-48e3-9165-28bb1f7adf0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:15:11.701868 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c04214-8ede-48e3-9165-28bb1f7adf0b service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:15:11.705456 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:15:11.724209 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 759/1505] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:15:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:15:11.771157 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94a6ad-9827-4000-8a13-3f422c2ef984 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:15:11.771700 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94a6ad-9827-4000-8a13-3f422c2ef984 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:15:11.771985 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94a6ad-9827-4000-8a13-3f422c2ef984 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:15:11.773249 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 747/1506] 10.222.0.16 () {52 vars in 830 bytes} [Mon Sep 5 12:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:15:11.777301 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c04214-8ede-48e3-9165-28bb1f7adf0b service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:15:11.778565 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-10c04214-8ede-48e3-9165-28bb1f7adf0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:15:11.877866 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c04214-8ede-48e3-9165-28bb1f7adf0b service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:15:11.970586 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2adab8-59aa-4fd4-9239-37193ee4adee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:15:11.971286 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2adab8-59aa-4fd4-9239-37193ee4adee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:15:11.971396 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2adab8-59aa-4fd4-9239-37193ee4adee admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:15:11.971666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2adab8-59aa-4fd4-9239-37193ee4adee admin admin] PATH_INFO: `/v3/users/dc2636095ab84f23b58023da9b207270` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:15:11.983891 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2adab8-59aa-4fd4-9239-37193ee4adee admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc2636095ab84f23b58023da9b207270)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:15:11.984920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: WARNING py.warnings [None req-5e2adab8-59aa-4fd4-9239-37193ee4adee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 12:15:11.984920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: warnings.warn(msg) Sep 05 12:15:11.984920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: Sep 05 12:15:11.986127 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2adab8-59aa-4fd4-9239-37193ee4adee admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:15:11.997029 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:15:12.055329 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-5e2adab8-59aa-4fd4-9239-37193ee4adee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc2636095ab84f23b58023da9b207270'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:15:12.071370 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-5e2adab8-59aa-4fd4-9239-37193ee4adee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc2636095ab84f23b58023da9b207270'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:15:12.071884 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-5e2adab8-59aa-4fd4-9239-37193ee4adee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc2636095ab84f23b58023da9b207270'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:15:12.072432 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-5e2adab8-59aa-4fd4-9239-37193ee4adee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc2636095ab84f23b58023da9b207270'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:15:12.095205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-5e2adab8-59aa-4fd4-9239-37193ee4adee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc2636095ab84f23b58023da9b207270'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:15:12.110007 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 760/1507] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:15:11 2022] DELETE /identity/v3/users/dc2636095ab84f23b58023da9b207270 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:15:12.115620 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a9bcf-2e25-44ee-a358-fff14034d2bf None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:15:12.115963 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a9bcf-2e25-44ee-a358-fff14034d2bf None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:15:12.116328 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a9bcf-2e25-44ee-a358-fff14034d2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:15:12.176607 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-777a9bcf-2e25-44ee-a358-fff14034d2bf None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:15:12.177825 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-777a9bcf-2e25-44ee-a358-fff14034d2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:15:12.255700 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:15:12.268392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 748/1508] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:15:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:15:12.278451 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2adab8-59aa-4fd4-9239-37193ee4adee admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:15:12.279739 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5e2adab8-59aa-4fd4-9239-37193ee4adee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:15:12.393107 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2adab8-59aa-4fd4-9239-37193ee4adee admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:15:12.449313 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:15:12.470539 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8128bf2-d5f2-47a7-8e7e-542fbe2a2f26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:15:12.471186 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8128bf2-d5f2-47a7-8e7e-542fbe2a2f26 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:15:12.471462 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8128bf2-d5f2-47a7-8e7e-542fbe2a2f26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:15:12.471774 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8128bf2-d5f2-47a7-8e7e-542fbe2a2f26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:15:12.473862 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d8128bf2-d5f2-47a7-8e7e-542fbe2a2f26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:15:12.582962 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8128bf2-d5f2-47a7-8e7e-542fbe2a2f26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:15:12.584447 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8128bf2-d5f2-47a7-8e7e-542fbe2a2f26 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:15:12.603903 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 761/1509] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:15:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:15:12.953485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a9bcf-2e25-44ee-a358-fff14034d2bf None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:15:12.954721 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-777a9bcf-2e25-44ee-a358-fff14034d2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:15:12.967653 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a9bcf-2e25-44ee-a358-fff14034d2bf None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:15:13.069279 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0893042-b56a-4373-9719-5a33a9776457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:15:13.069903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0893042-b56a-4373-9719-5a33a9776457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:15:13.070177 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0893042-b56a-4373-9719-5a33a9776457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:15:13.070442 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0893042-b56a-4373-9719-5a33a9776457 admin admin] PATH_INFO: `/v3/projects/c36b205827634232ac0b6b89e597c33d` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:15:13.080711 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0893042-b56a-4373-9719-5a33a9776457 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c36b205827634232ac0b6b89e597c33d)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:15:13.082383 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: WARNING py.warnings [None req-a0893042-b56a-4373-9719-5a33a9776457 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 12:15:13.082383 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: warnings.warn(msg) Sep 05 12:15:13.082383 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: Sep 05 12:15:13.083610 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0893042-b56a-4373-9719-5a33a9776457 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:15:13.143736 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-a0893042-b56a-4373-9719-5a33a9776457 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c36b205827634232ac0b6b89e597c33d', 'request_id': 'req-a0893042-b56a-4373-9719-5a33a9776457'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:15:13.154030 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-a0893042-b56a-4373-9719-5a33a9776457 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c36b205827634232ac0b6b89e597c33d', 'request_id': 'req-a0893042-b56a-4373-9719-5a33a9776457'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:15:13.160744 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-a0893042-b56a-4373-9719-5a33a9776457 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c36b205827634232ac0b6b89e597c33d', 'request_id': 'req-a0893042-b56a-4373-9719-5a33a9776457'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:15:13.191440 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-a0893042-b56a-4373-9719-5a33a9776457 admin admin] The token cache is being invalidate because project c36b205827634232ac0b6b89e597c33d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:15:13.191761 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-a0893042-b56a-4373-9719-5a33a9776457 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:15:13.192298 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-a0893042-b56a-4373-9719-5a33a9776457 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:15:13.193632 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 749/1510] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:15:12 2022] DELETE /identity/v3/projects/c36b205827634232ac0b6b89e597c33d => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:15:22.601528 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=62 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:15:23.188808 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:16:41.060069 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ab565-084d-461d-b7f0-5876c06ce98f None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:16:41.060686 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ab565-084d-461d-b7f0-5876c06ce98f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:16:41.060686 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ab565-084d-461d-b7f0-5876c06ce98f None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:16:41.061985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 762/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 12:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:16:41.067306 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cfabf6-06b8-4b63-91be-913c7689a68c None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:16:41.067912 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cfabf6-06b8-4b63-91be-913c7689a68c None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:16:41.067912 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cfabf6-06b8-4b63-91be-913c7689a68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:16:41.161548 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-29cfabf6-06b8-4b63-91be-913c7689a68c None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:16:41.162789 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-29cfabf6-06b8-4b63-91be-913c7689a68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:16:41.277124 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 750/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 12:16:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:16:41.432337 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ab565-084d-461d-b7f0-5876c06ce98f None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:16:41.434487 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b18ab565-084d-461d-b7f0-5876c06ce98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:16:41.555745 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ab565-084d-461d-b7f0-5876c06ce98f None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:16:41.561407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2feebc-a461-4cc0-8d1f-efc5228b9616 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:16:41.562010 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2feebc-a461-4cc0-8d1f-efc5228b9616 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:16:41.562288 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2feebc-a461-4cc0-8d1f-efc5228b9616 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:16:41.562561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2feebc-a461-4cc0-8d1f-efc5228b9616 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:16:41.572021 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2feebc-a461-4cc0-8d1f-efc5228b9616 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:16:41.573619 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2feebc-a461-4cc0-8d1f-efc5228b9616 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:16:41.583969 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 763/1513] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:16:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:16:42.423511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29da97-4c3c-4ebe-be1d-53ac3e77ef46 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:16:42.424381 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29da97-4c3c-4ebe-be1d-53ac3e77ef46 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:16:42.424555 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29da97-4c3c-4ebe-be1d-53ac3e77ef46 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:16:42.425997 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 751/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 12:16:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:16:42.431032 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771fd3e7-091f-4c7d-856a-74119a2723be None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:16:42.431489 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771fd3e7-091f-4c7d-856a-74119a2723be None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:16:42.431630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771fd3e7-091f-4c7d-856a-74119a2723be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:16:42.451399 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-771fd3e7-091f-4c7d-856a-74119a2723be None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:16:42.452670 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-771fd3e7-091f-4c7d-856a-74119a2723be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:16:42.461383 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 764/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 12:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:16:42.614726 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29da97-4c3c-4ebe-be1d-53ac3e77ef46 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:16:42.629187 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29da97-4c3c-4ebe-be1d-53ac3e77ef46 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:16:42.634650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0a0c2-d6d2-4a3a-a957-f3cd0f9bd226 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:16:42.635240 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0a0c2-d6d2-4a3a-a957-f3cd0f9bd226 service nova] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:16:42.635650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0a0c2-d6d2-4a3a-a957-f3cd0f9bd226 service nova] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:16:42.635830 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0a0c2-d6d2-4a3a-a957-f3cd0f9bd226 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:16:42.644016 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0a0c2-d6d2-4a3a-a957-f3cd0f9bd226 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:16:42.646166 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0a0c2-d6d2-4a3a-a957-f3cd0f9bd226 service nova] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:16:42.656123 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 752/1516] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:16:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:16:43.225698 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f27e8-9854-44f6-8892-6c61606326f6 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:16:43.225978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f27e8-9854-44f6-8892-6c61606326f6 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:16:43.226252 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f27e8-9854-44f6-8892-6c61606326f6 None None] PATH_INFO: `/v3` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:16:43.227561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 765/1517] 10.222.0.55 () {54 vars in 925 bytes} [Mon Sep 5 12:16:43 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:16:43.234563 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798b82d-1359-4e9f-918a-eaf1f72ac3eb None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:16:43.234884 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798b82d-1359-4e9f-918a-eaf1f72ac3eb None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:16:43.235164 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798b82d-1359-4e9f-918a-eaf1f72ac3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:16:43.296706 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-7798b82d-1359-4e9f-918a-eaf1f72ac3eb None None] MFA Rules not processed for user `524bcfffcdfa4790885e0d4cbd66fa80`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:16:43.297929 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-7798b82d-1359-4e9f-918a-eaf1f72ac3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:16:43.314938 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:16:43.371626 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 753/1518] 10.222.0.55 () {58 vars in 1015 bytes} [Mon Sep 5 12:16:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:16:52.446343 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=56 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:16:53.368608 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=23 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:16:53.743460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81c90e-e3d9-4be9-90b9-d4390621a9d7 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:16:53.744367 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81c90e-e3d9-4be9-90b9-d4390621a9d7 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:16:53.744367 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81c90e-e3d9-4be9-90b9-d4390621a9d7 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:16:53.745688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 766/1519] 10.222.0.16 () {54 vars in 961 bytes} [Mon Sep 5 12:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:16:53.751260 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab18a6-4fdb-4ec0-8dbb-de9a655340c6 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:16:53.751626 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab18a6-4fdb-4ec0-8dbb-de9a655340c6 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:16:53.751895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab18a6-4fdb-4ec0-8dbb-de9a655340c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:16:53.774113 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-adab18a6-4fdb-4ec0-8dbb-de9a655340c6 None None] MFA Rules not processed for user `524bcfffcdfa4790885e0d4cbd66fa80`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:16:53.775627 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-adab18a6-4fdb-4ec0-8dbb-de9a655340c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:16:53.784903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 754/1520] 10.222.0.16 () {58 vars in 1058 bytes} [Mon Sep 5 12:16:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:16:53.793745 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81c90e-e3d9-4be9-90b9-d4390621a9d7 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:16:53.817909 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81c90e-e3d9-4be9-90b9-d4390621a9d7 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:16:53.826745 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a14e38-776b-4ca5-b07f-e5bd3f2c3cda service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524bcfffcdfa4790885e0d4cbd66fa80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:16:53.827825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a14e38-776b-4ca5-b07f-e5bd3f2c3cda service placement] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:16:53.828324 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a14e38-776b-4ca5-b07f-e5bd3f2c3cda service placement] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:16:53.828791 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a14e38-776b-4ca5-b07f-e5bd3f2c3cda service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:16:53.844779 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a14e38-776b-4ca5-b07f-e5bd3f2c3cda service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:16:53.847834 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a14e38-776b-4ca5-b07f-e5bd3f2c3cda service placement] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:16:53.867306 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 767/1521] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:16:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:16:54.037231 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cca61-9e23-4c14-ab4e-e48b9b77b163 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:16:54.037534 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cca61-9e23-4c14-ab4e-e48b9b77b163 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:16:54.037778 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cca61-9e23-4c14-ab4e-e48b9b77b163 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:16:54.039404 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 755/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 12:16:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:16:54.045420 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96315aa9-a281-4c27-aa5f-bd6c59f67653 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:16:54.045711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96315aa9-a281-4c27-aa5f-bd6c59f67653 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:16:54.046017 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96315aa9-a281-4c27-aa5f-bd6c59f67653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:16:54.067779 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-96315aa9-a281-4c27-aa5f-bd6c59f67653 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:16:54.069011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-96315aa9-a281-4c27-aa5f-bd6c59f67653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:16:54.077049 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 768/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 12:16:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:16:54.238072 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cca61-9e23-4c14-ab4e-e48b9b77b163 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:16:54.259855 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cca61-9e23-4c14-ab4e-e48b9b77b163 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:16:54.266140 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2cd091-42c3-4624-82b5-7f6c97abdd5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:16:54.266745 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cd091-42c3-4624-82b5-7f6c97abdd5b service nova] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:16:54.267023 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cd091-42c3-4624-82b5-7f6c97abdd5b service nova] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:16:54.267297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cd091-42c3-4624-82b5-7f6c97abdd5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:16:54.276678 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2cd091-42c3-4624-82b5-7f6c97abdd5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:16:54.278294 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2cd091-42c3-4624-82b5-7f6c97abdd5b service nova] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:16:54.290032 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 756/1524] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:16:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:16:55.146192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64114aad-f0f3-46d7-8419-fbd4682550ef None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:16:55.146858 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64114aad-f0f3-46d7-8419-fbd4682550ef None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:16:55.146858 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64114aad-f0f3-46d7-8419-fbd4682550ef None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:16:55.148129 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 769/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 12:16:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:16:55.153903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251312d3-ec9f-4dfe-a31e-15feeeb18618 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:16:55.154567 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251312d3-ec9f-4dfe-a31e-15feeeb18618 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:16:55.154567 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251312d3-ec9f-4dfe-a31e-15feeeb18618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:16:55.177832 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-251312d3-ec9f-4dfe-a31e-15feeeb18618 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:16:55.179456 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-251312d3-ec9f-4dfe-a31e-15feeeb18618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:16:55.187733 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 757/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 12:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:16:55.342789 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64114aad-f0f3-46d7-8419-fbd4682550ef None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:16:55.358955 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64114aad-f0f3-46d7-8419-fbd4682550ef None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:16:55.368145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff1e75e-128e-49a6-ac03-fc7a038cc630 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:16:55.369154 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1e75e-128e-49a6-ac03-fc7a038cc630 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:16:55.369623 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1e75e-128e-49a6-ac03-fc7a038cc630 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:16:55.370105 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1e75e-128e-49a6-ac03-fc7a038cc630 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:16:55.383963 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff1e75e-128e-49a6-ac03-fc7a038cc630 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:16:55.385566 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff1e75e-128e-49a6-ac03-fc7a038cc630 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:16:55.395811 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 770/1527] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:16:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:16:59.567007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9feeeb-e1a2-427e-8b5d-df37fc05780b None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:16:59.567931 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9feeeb-e1a2-427e-8b5d-df37fc05780b None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:16:59.568619 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9feeeb-e1a2-427e-8b5d-df37fc05780b None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:16:59.571947 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 758/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 12:16:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:16:59.578795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1800d-9b89-4d95-9d43-59dbf76f2deb None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:16:59.579366 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1800d-9b89-4d95-9d43-59dbf76f2deb None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:16:59.579835 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1800d-9b89-4d95-9d43-59dbf76f2deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:16:59.603616 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-1fd1800d-9b89-4d95-9d43-59dbf76f2deb None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:16:59.604925 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1fd1800d-9b89-4d95-9d43-59dbf76f2deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:16:59.614182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 771/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 12:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:16:59.925291 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9feeeb-e1a2-427e-8b5d-df37fc05780b None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:16:59.927999 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-7c9feeeb-e1a2-427e-8b5d-df37fc05780b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:17:00.029957 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9feeeb-e1a2-427e-8b5d-df37fc05780b None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:17:00.034156 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928b06c-a8b8-4331-935b-70906f2103c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:17:00.034947 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928b06c-a8b8-4331-935b-70906f2103c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:17:00.035408 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928b06c-a8b8-4331-935b-70906f2103c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:17:00.035812 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928b06c-a8b8-4331-935b-70906f2103c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:17:00.046887 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9928b06c-a8b8-4331-935b-70906f2103c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:17:00.048600 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9928b06c-a8b8-4331-935b-70906f2103c7 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:17:00.059913 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 759/1530] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:16:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:17:02.115034 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e7811-b0f5-4910-a1ea-f044af45148f None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:17:02.115778 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e7811-b0f5-4910-a1ea-f044af45148f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:17:02.115778 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e7811-b0f5-4910-a1ea-f044af45148f None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:17:02.117254 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 772/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 12:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:17:02.123797 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a6c30-6799-4f94-8e49-efcc14aee1a4 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:17:02.124415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a6c30-6799-4f94-8e49-efcc14aee1a4 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:17:02.124415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a6c30-6799-4f94-8e49-efcc14aee1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:17:02.147479 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-043a6c30-6799-4f94-8e49-efcc14aee1a4 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:17:02.149107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-043a6c30-6799-4f94-8e49-efcc14aee1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:17:02.159182 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 760/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 12:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:17:02.459437 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e7811-b0f5-4910-a1ea-f044af45148f None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:17:02.471432 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e7811-b0f5-4910-a1ea-f044af45148f None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:17:02.475361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4b98c-e3ae-4097-86c4-f4fc4b8a1908 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:17:02.476065 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4b98c-e3ae-4097-86c4-f4fc4b8a1908 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:17:02.476379 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4b98c-e3ae-4097-86c4-f4fc4b8a1908 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:17:02.476685 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4b98c-e3ae-4097-86c4-f4fc4b8a1908 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:17:02.487803 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4b98c-e3ae-4097-86c4-f4fc4b8a1908 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:17:02.489400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4b98c-e3ae-4097-86c4-f4fc4b8a1908 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:17:02.500923 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 773/1533] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:17:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:17:06.004755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30673333-1d72-41f5-b4ce-fcfefb2d3b9a None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:17:06.005633 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30673333-1d72-41f5-b4ce-fcfefb2d3b9a None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:17:06.005633 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30673333-1d72-41f5-b4ce-fcfefb2d3b9a None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:17:06.006668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 761/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 12:17:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:17:06.012580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc47c48-581b-4cc2-988c-b5ab4f9de665 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:17:06.013425 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc47c48-581b-4cc2-988c-b5ab4f9de665 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:17:06.013425 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc47c48-581b-4cc2-988c-b5ab4f9de665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:17:06.035540 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-3cc47c48-581b-4cc2-988c-b5ab4f9de665 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:17:06.036836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3cc47c48-581b-4cc2-988c-b5ab4f9de665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:17:06.045440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 774/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 12:17:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:17:06.648097 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30673333-1d72-41f5-b4ce-fcfefb2d3b9a None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:17:06.650357 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-30673333-1d72-41f5-b4ce-fcfefb2d3b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:17:06.767609 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30673333-1d72-41f5-b4ce-fcfefb2d3b9a None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:17:06.771605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c746f9-452a-4a20-8858-42763e1ead9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:17:06.772258 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c746f9-452a-4a20-8858-42763e1ead9e service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:17:06.772532 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c746f9-452a-4a20-8858-42763e1ead9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:17:06.772799 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c746f9-452a-4a20-8858-42763e1ead9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:17:06.783217 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c746f9-452a-4a20-8858-42763e1ead9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:17:06.784732 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c746f9-452a-4a20-8858-42763e1ead9e service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:17:06.796604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 762/1536] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:17:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:17:08.872153 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186575c-e6c7-4b39-ae75-4478d1af9d64 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:17:08.872868 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186575c-e6c7-4b39-ae75-4478d1af9d64 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:17:08.872868 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186575c-e6c7-4b39-ae75-4478d1af9d64 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:17:08.874064 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 775/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 12:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:17:08.881170 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45838b48-7887-4ded-a7d6-545f949975c8 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:17:08.881803 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45838b48-7887-4ded-a7d6-545f949975c8 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:17:08.881803 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45838b48-7887-4ded-a7d6-545f949975c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:17:08.905270 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-45838b48-7887-4ded-a7d6-545f949975c8 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:17:08.906700 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-45838b48-7887-4ded-a7d6-545f949975c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:17:08.916086 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 763/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 12:17:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:17:09.967448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186575c-e6c7-4b39-ae75-4478d1af9d64 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:17:09.970220 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7186575c-e6c7-4b39-ae75-4478d1af9d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:17:09.994695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186575c-e6c7-4b39-ae75-4478d1af9d64 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:17:10.001025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba25e6-0f45-4ff7-953e-e6ef495e9602 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:17:10.002068 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba25e6-0f45-4ff7-953e-e6ef495e9602 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:17:10.002557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba25e6-0f45-4ff7-953e-e6ef495e9602 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:17:10.003030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba25e6-0f45-4ff7-953e-e6ef495e9602 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:17:10.020651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba25e6-0f45-4ff7-953e-e6ef495e9602 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:17:10.022170 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba25e6-0f45-4ff7-953e-e6ef495e9602 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:17:10.034138 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 776/1539] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:17:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:17:18.898233 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=88 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:17:20.029248 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=33 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:39.154224 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282bc61-94c5-43eb-965c-03d514d09d81 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:39.155379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282bc61-94c5-43eb-965c-03d514d09d81 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:39.155379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282bc61-94c5-43eb-965c-03d514d09d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:39.206375 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-a282bc61-94c5-43eb-965c-03d514d09d81 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:39.208543 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-a282bc61-94c5-43eb-965c-03d514d09d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:39.229948 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 764/1540] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:39.239566 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9218b-5e28-4cf6-96e6-698d1207c397 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:39.240172 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9218b-5e28-4cf6-96e6-698d1207c397 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:39.240530 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9218b-5e28-4cf6-96e6-698d1207c397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:39.289177 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-90d9218b-5e28-4cf6-96e6-698d1207c397 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:39.291349 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-90d9218b-5e28-4cf6-96e6-698d1207c397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:39.317408 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f7413-b891-4628-b638-86bf3df5b5f8 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:39.317885 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f7413-b891-4628-b638-86bf3df5b5f8 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:39.318362 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f7413-b891-4628-b638-86bf3df5b5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:39.335643 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 777/1541] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:39.337325 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9218b-5e28-4cf6-96e6-698d1207c397 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:39.380263 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9218b-5e28-4cf6-96e6-698d1207c397 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:39.388362 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-f56f7413-b891-4628-b638-86bf3df5b5f8 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:39.390470 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-f56f7413-b891-4628-b638-86bf3df5b5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:39.401004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f58eb4-c732-4ff4-9727-18aa98e3eeef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:39.402088 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f58eb4-c732-4ff4-9727-18aa98e3eeef admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:39.402584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f58eb4-c732-4ff4-9727-18aa98e3eeef admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:39.403075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f58eb4-c732-4ff4-9727-18aa98e3eeef admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:39.404388 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f58eb4-c732-4ff4-9727-18aa98e3eeef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:18:39.404881 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f58eb4-c732-4ff4-9727-18aa98e3eeef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:39.406868 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: WARNING py.warnings [None req-17f58eb4-c732-4ff4-9727-18aa98e3eeef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 05 12:18:39.406868 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: warnings.warn(msg) Sep 05 12:18:39.406868 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: Sep 05 12:18:39.408812 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f58eb4-c732-4ff4-9727-18aa98e3eeef admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:39.425993 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 765/1542] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:39.438617 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b123c-a724-48de-b29a-4c856838081d None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:39.440051 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b123c-a724-48de-b29a-4c856838081d None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:39.440782 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b123c-a724-48de-b29a-4c856838081d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:39.477566 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 778/1543] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:18:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:18:39.493372 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f58eb4-c732-4ff4-9727-18aa98e3eeef admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:39.523926 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-024b123c-a724-48de-b29a-4c856838081d None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:39.528433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f58eb4-c732-4ff4-9727-18aa98e3eeef admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:39.531072 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-024b123c-a724-48de-b29a-4c856838081d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:39.555144 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0998c68-475c-48ba-9496-8f54b58ad992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:39.556835 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0998c68-475c-48ba-9496-8f54b58ad992 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:39.558327 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0998c68-475c-48ba-9496-8f54b58ad992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:39.560314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0998c68-475c-48ba-9496-8f54b58ad992 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:39.561775 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0998c68-475c-48ba-9496-8f54b58ad992 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:39.565882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 766/1544] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:39.572344 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0998c68-475c-48ba-9496-8f54b58ad992 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:39.574397 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b123c-a724-48de-b29a-4c856838081d None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:39.618982 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b123c-a724-48de-b29a-4c856838081d None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:39.643558 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c69f54-3c24-4a0f-a2d3-8e617543bf22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:39.643558 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c69f54-3c24-4a0f-a2d3-8e617543bf22 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:39.643558 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c69f54-3c24-4a0f-a2d3-8e617543bf22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:39.643558 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c69f54-3c24-4a0f-a2d3-8e617543bf22 admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:39.644338 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c69f54-3c24-4a0f-a2d3-8e617543bf22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:18:39.644338 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c69f54-3c24-4a0f-a2d3-8e617543bf22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:39.651017 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c69f54-3c24-4a0f-a2d3-8e617543bf22 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:39.687283 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 767/1545] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:18:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:18:39.698425 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c69f54-3c24-4a0f-a2d3-8e617543bf22 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:39.717780 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 779/1546] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:18:39 2022] POST /identity/v3/projects => generated 371 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:18:39.724136 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0998c68-475c-48ba-9496-8f54b58ad992 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:39.728019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c69f54-3c24-4a0f-a2d3-8e617543bf22 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:39.888830 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac7984-b2de-4d70-a866-58630d39e580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:39.889825 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac7984-b2de-4d70-a866-58630d39e580 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:39.890310 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac7984-b2de-4d70-a866-58630d39e580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:39.890989 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac7984-b2de-4d70-a866-58630d39e580 admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:39.892409 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac7984-b2de-4d70-a866-58630d39e580 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:39.894641 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: WARNING py.warnings [None req-8cac7984-b2de-4d70-a866-58630d39e580 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 12:18:39.894641 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: warnings.warn(msg) Sep 05 12:18:39.894641 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: Sep 05 12:18:39.896961 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac7984-b2de-4d70-a866-58630d39e580 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:39.911142 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0998c68-475c-48ba-9496-8f54b58ad992 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:39.920813 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc4f67-b346-44c1-b322-e29ae82713d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:39.921475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc4f67-b346-44c1-b322-e29ae82713d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:39.921953 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc4f67-b346-44c1-b322-e29ae82713d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:39.922158 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc4f67-b346-44c1-b322-e29ae82713d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:39.922904 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfc4f67-b346-44c1-b322-e29ae82713d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:39.924964 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: WARNING py.warnings [None req-dbfc4f67-b346-44c1-b322-e29ae82713d6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 12:18:39.924964 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: warnings.warn(msg) Sep 05 12:18:39.924964 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: Sep 05 12:18:39.926437 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfc4f67-b346-44c1-b322-e29ae82713d6 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:39.967267 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 768/1547] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:18:39 2022] POST /identity/v3/projects => generated 389 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:18:39.968909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a053ec85-5b60-4d8a-8dea-4ad212018747 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:39.969278 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a053ec85-5b60-4d8a-8dea-4ad212018747 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:39.972169 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a053ec85-5b60-4d8a-8dea-4ad212018747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:39.974919 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 780/1548] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:18:39 2022] POST /identity/v3/users => generated 286 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:18:39.979554 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc4f67-b346-44c1-b322-e29ae82713d6 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:40.015369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc4f67-b346-44c1-b322-e29ae82713d6 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:40.030625 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-a053ec85-5b60-4d8a-8dea-4ad212018747 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:40.031947 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-a053ec85-5b60-4d8a-8dea-4ad212018747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:40.145088 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f2dae-68a1-4772-a132-5a9b4ea6ef5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:40.145840 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f2dae-68a1-4772-a132-5a9b4ea6ef5f admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:40.146119 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f2dae-68a1-4772-a132-5a9b4ea6ef5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:40.146397 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f2dae-68a1-4772-a132-5a9b4ea6ef5f admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:40.147077 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876f2dae-68a1-4772-a132-5a9b4ea6ef5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:40.150009 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876f2dae-68a1-4772-a132-5a9b4ea6ef5f admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:40.210161 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 769/1549] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:40.212250 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a053ec85-5b60-4d8a-8dea-4ad212018747 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:40.223637 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 781/1550] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:18:39 2022] POST /identity/v3/users => generated 295 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:18:40.224979 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2ff8c-2853-4d8c-a90e-4f7ef08d7ddc None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:40.225583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2ff8c-2853-4d8c-a90e-4f7ef08d7ddc None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:40.225920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2ff8c-2853-4d8c-a90e-4f7ef08d7ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:40.240309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a053ec85-5b60-4d8a-8dea-4ad212018747 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:40.260888 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae8775-813e-45c8-b906-88e03e439e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:40.261907 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae8775-813e-45c8-b906-88e03e439e01 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:40.262394 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae8775-813e-45c8-b906-88e03e439e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:40.262878 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae8775-813e-45c8-b906-88e03e439e01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:40.264138 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ae8775-813e-45c8-b906-88e03e439e01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:40.268182 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ae8775-813e-45c8-b906-88e03e439e01 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:40.291245 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-57e2ff8c-2853-4d8c-a90e-4f7ef08d7ddc None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:40.292537 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-57e2ff8c-2853-4d8c-a90e-4f7ef08d7ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:40.306226 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 770/1551] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:18:40.308749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f023db7-1c58-49da-b367-90a57b9e6631 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:40.309229 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f023db7-1c58-49da-b367-90a57b9e6631 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:40.309714 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f023db7-1c58-49da-b367-90a57b9e6631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:40.317496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 782/1552] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:40.318480 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2ff8c-2853-4d8c-a90e-4f7ef08d7ddc None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:40.338710 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2ff8c-2853-4d8c-a90e-4f7ef08d7ddc None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:40.350343 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0dc87b-fd3c-424b-97fe-f8ba52926314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:40.350973 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dc87b-fd3c-424b-97fe-f8ba52926314 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:40.351252 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dc87b-fd3c-424b-97fe-f8ba52926314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:40.351505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dc87b-fd3c-424b-97fe-f8ba52926314 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:40.352183 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0dc87b-fd3c-424b-97fe-f8ba52926314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:40.353789 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0dc87b-fd3c-424b-97fe-f8ba52926314 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:40.372488 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-6f023db7-1c58-49da-b367-90a57b9e6631 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:40.374477 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-6f023db7-1c58-49da-b367-90a57b9e6631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:40.384049 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 783/1553] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:18:40.385134 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0dc87b-fd3c-424b-97fe-f8ba52926314 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:40.410812 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0dc87b-fd3c-424b-97fe-f8ba52926314 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:40.413833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 771/1554] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:40.415992 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba934b3-9fe4-47ed-8835-992d8948e49a None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:40.416460 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba934b3-9fe4-47ed-8835-992d8948e49a None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:40.416907 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba934b3-9fe4-47ed-8835-992d8948e49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:40.421043 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b4e6a-d6d3-48a1-a0de-f19992e06cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:40.421726 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b4e6a-d6d3-48a1-a0de-f19992e06cf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:40.422014 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b4e6a-d6d3-48a1-a0de-f19992e06cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:40.422266 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b4e6a-d6d3-48a1-a0de-f19992e06cf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:40.422921 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b4e6a-d6d3-48a1-a0de-f19992e06cf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:40.424668 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b4e6a-d6d3-48a1-a0de-f19992e06cf5 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:40.437851 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 784/1555] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:18:40.438661 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b4e6a-d6d3-48a1-a0de-f19992e06cf5 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:40.448072 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:40.476579 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b4e6a-d6d3-48a1-a0de-f19992e06cf5 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:40.488741 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-2ba934b3-9fe4-47ed-8835-992d8948e49a None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:40.490581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-2ba934b3-9fe4-47ed-8835-992d8948e49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:40.505249 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841476a-86d9-45e9-a7e6-3e8860382cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:40.506269 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841476a-86d9-45e9-a7e6-3e8860382cee admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:40.506747 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841476a-86d9-45e9-a7e6-3e8860382cee admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:40.507224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841476a-86d9-45e9-a7e6-3e8860382cee admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:40.508422 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841476a-86d9-45e9-a7e6-3e8860382cee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:40.511473 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841476a-86d9-45e9-a7e6-3e8860382cee admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:40.538636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 772/1556] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:40.543723 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba934b3-9fe4-47ed-8835-992d8948e49a None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:40.568397 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 785/1557] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:18:40.569940 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841476a-86d9-45e9-a7e6-3e8860382cee admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:40.570850 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba934b3-9fe4-47ed-8835-992d8948e49a None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:40.582350 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954a4e5-dc7b-4bc7-9a8a-c370e309822b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:40.582982 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954a4e5-dc7b-4bc7-9a8a-c370e309822b admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:40.583275 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954a4e5-dc7b-4bc7-9a8a-c370e309822b admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:40.583560 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954a4e5-dc7b-4bc7-9a8a-c370e309822b admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:40.584212 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954a4e5-dc7b-4bc7-9a8a-c370e309822b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:18:40.584518 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954a4e5-dc7b-4bc7-9a8a-c370e309822b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:40.586159 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954a4e5-dc7b-4bc7-9a8a-c370e309822b admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:40.596285 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841476a-86d9-45e9-a7e6-3e8860382cee admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:40.613482 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22cf7f-c01a-47c1-8025-d7685240770a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:40.614585 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22cf7f-c01a-47c1-8025-d7685240770a admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:40.615065 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22cf7f-c01a-47c1-8025-d7685240770a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:40.615564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22cf7f-c01a-47c1-8025-d7685240770a admin admin] PATH_INFO: `/v3/projects/47557a3ca7624ad9b227ea0e51fd414e/users/231eca85785c49628baaee7d6066c3f1/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:40.631104 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 773/1558] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:18:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:18:40.640265 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954a4e5-dc7b-4bc7-9a8a-c370e309822b admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:40.676147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954a4e5-dc7b-4bc7-9a8a-c370e309822b admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:40.677512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22cf7f-c01a-47c1-8025-d7685240770a admin admin] RBAC: Authorizing `identity:create_grant(project_id=47557a3ca7624ad9b227ea0e51fd414e, user_id=231eca85785c49628baaee7d6066c3f1, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:40.683472 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22cf7f-c01a-47c1-8025-d7685240770a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:40.696686 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a0e144-3039-4bb1-8b4f-36cf7fe1d2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:40.697709 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a0e144-3039-4bb1-8b4f-36cf7fe1d2c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:40.698186 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a0e144-3039-4bb1-8b4f-36cf7fe1d2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:40.698669 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a0e144-3039-4bb1-8b4f-36cf7fe1d2c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:40.701357 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a0e144-3039-4bb1-8b4f-36cf7fe1d2c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:18:40.701894 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a0e144-3039-4bb1-8b4f-36cf7fe1d2c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:40.706092 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a0e144-3039-4bb1-8b4f-36cf7fe1d2c4 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:40.716807 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 786/1559] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:18:40 2022] PUT /identity/v3/projects/47557a3ca7624ad9b227ea0e51fd414e/users/231eca85785c49628baaee7d6066c3f1/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:18:40.725028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33347d-c90a-49e9-afff-f8382cabfe4c None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:40.726681 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33347d-c90a-49e9-afff-f8382cabfe4c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:40.728101 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33347d-c90a-49e9-afff-f8382cabfe4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:40.737771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 774/1560] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:18:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:18:40.738728 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a0e144-3039-4bb1-8b4f-36cf7fe1d2c4 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:40.764844 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:40.777566 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a0e144-3039-4bb1-8b4f-36cf7fe1d2c4 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:40.779275 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-bf33347d-c90a-49e9-afff-f8382cabfe4c None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:40.781596 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-bf33347d-c90a-49e9-afff-f8382cabfe4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:40.868927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 787/1561] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:40.869802 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf33347d-c90a-49e9-afff-f8382cabfe4c None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:40.881489 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf33347d-c90a-49e9-afff-f8382cabfe4c None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:40.886068 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee7007-cadb-401d-b658-d8bbc8a1436f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:40.886480 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7051057-583c-4833-b3fe-f6296cbd4caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:40.886773 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee7007-cadb-401d-b658-d8bbc8a1436f admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:40.887051 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee7007-cadb-401d-b658-d8bbc8a1436f admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:40.887161 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7051057-583c-4833-b3fe-f6296cbd4caa admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:40.887326 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee7007-cadb-401d-b658-d8bbc8a1436f admin admin] PATH_INFO: `/v3/projects/ca5b70833b074d199437d3f10e511306/users/83d3b245c185424a9b6e5d69d691f031/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:40.887431 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7051057-583c-4833-b3fe-f6296cbd4caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:40.887650 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7051057-583c-4833-b3fe-f6296cbd4caa admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:40.888364 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7051057-583c-4833-b3fe-f6296cbd4caa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:40.890282 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7051057-583c-4833-b3fe-f6296cbd4caa admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:40.905222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee7007-cadb-401d-b658-d8bbc8a1436f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca5b70833b074d199437d3f10e511306, user_id=83d3b245c185424a9b6e5d69d691f031, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:40.906295 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: WARNING py.warnings [None req-ecee7007-cadb-401d-b658-d8bbc8a1436f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 12:18:40.906295 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: warnings.warn(msg) Sep 05 12:18:40.906295 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: Sep 05 12:18:40.908393 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee7007-cadb-401d-b658-d8bbc8a1436f admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:40.924913 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 775/1562] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:18:40 2022] PUT /identity/v3/projects/ca5b70833b074d199437d3f10e511306/users/83d3b245c185424a9b6e5d69d691f031/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:18:40.925742 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee7007-cadb-401d-b658-d8bbc8a1436f admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:40.932703 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 788/1563] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:18:40 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:18:40.934100 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039622b-ce15-45c2-aa2e-c8c984ae270c None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:40.934380 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039622b-ce15-45c2-aa2e-c8c984ae270c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:40.934661 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039622b-ce15-45c2-aa2e-c8c984ae270c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:40.966621 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-a039622b-ce15-45c2-aa2e-c8c984ae270c None None] MFA Rules not processed for user `231eca85785c49628baaee7d6066c3f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:40.967913 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a039622b-ce15-45c2-aa2e-c8c984ae270c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:41.011777 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee7007-cadb-401d-b658-d8bbc8a1436f admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:41.056641 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 789/1564] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:41.065027 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8a90c-7ad7-47c7-ae5b-8d54b245f73c None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:41.065027 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8a90c-7ad7-47c7-ae5b-8d54b245f73c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:41.065027 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8a90c-7ad7-47c7-ae5b-8d54b245f73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:41.090550 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-07b8a90c-7ad7-47c7-ae5b-8d54b245f73c None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:41.093197 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-07b8a90c-7ad7-47c7-ae5b-8d54b245f73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:41.120757 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa85f9df-9e11-4433-8e77-27fdeeb38076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:41.121554 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85f9df-9e11-4433-8e77-27fdeeb38076 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:41.122052 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85f9df-9e11-4433-8e77-27fdeeb38076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:41.122415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85f9df-9e11-4433-8e77-27fdeeb38076 admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:41.123563 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa85f9df-9e11-4433-8e77-27fdeeb38076 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:41.125668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa85f9df-9e11-4433-8e77-27fdeeb38076 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:41.137310 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:41.163621 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 776/1565] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:18:40 2022] POST /identity/v3/projects => generated 379 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:18:41.165876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36683a98-d780-4cf6-9a7d-adafd3d1e037 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:41.166196 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36683a98-d780-4cf6-9a7d-adafd3d1e037 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:41.166677 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36683a98-d780-4cf6-9a7d-adafd3d1e037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:41.176384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 790/1566] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:41.177889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72baf434-b9c2-46b9-afcf-89b44548bc8f None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:41.178192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72baf434-b9c2-46b9-afcf-89b44548bc8f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:41.178514 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72baf434-b9c2-46b9-afcf-89b44548bc8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:41.199794 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-72baf434-b9c2-46b9-afcf-89b44548bc8f None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:41.200262 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-36683a98-d780-4cf6-9a7d-adafd3d1e037 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:41.201037 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-72baf434-b9c2-46b9-afcf-89b44548bc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:41.202387 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-36683a98-d780-4cf6-9a7d-adafd3d1e037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:41.222576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:41.299896 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 777/1567] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:41.300167 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 791/1568] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:41.300628 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8065b-a832-447c-8598-ae342f292bff None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:41.300628 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8065b-a832-447c-8598-ae342f292bff None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:41.300628 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8065b-a832-447c-8598-ae342f292bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:41.300906 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77522e4-61da-4988-b8be-ad404b9381aa None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:41.300906 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77522e4-61da-4988-b8be-ad404b9381aa None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:41.300906 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77522e4-61da-4988-b8be-ad404b9381aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:41.322513 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-f77522e4-61da-4988-b8be-ad404b9381aa None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:41.323813 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f77522e4-61da-4988-b8be-ad404b9381aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:41.332231 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 792/1569] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:41.336685 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77522e4-61da-4988-b8be-ad404b9381aa None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:41.337067 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-d1e8065b-a832-447c-8598-ae342f292bff None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:41.339218 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-d1e8065b-a832-447c-8598-ae342f292bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:41.348071 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77522e4-61da-4988-b8be-ad404b9381aa None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:41.353165 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4439eb-1d56-4876-8583-6303a2ba1ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:41.353798 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4439eb-1d56-4876-8583-6303a2ba1ca4 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:41.354080 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4439eb-1d56-4876-8583-6303a2ba1ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:41.354354 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4439eb-1d56-4876-8583-6303a2ba1ca4 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:41.355038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4439eb-1d56-4876-8583-6303a2ba1ca4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:41.357369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4439eb-1d56-4876-8583-6303a2ba1ca4 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:41.357604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 778/1570] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:41.359808 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd9650-1a3f-41e3-a970-a2212c87442a None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:41.360182 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd9650-1a3f-41e3-a970-a2212c87442a None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:41.360682 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd9650-1a3f-41e3-a970-a2212c87442a None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:41.364343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 779/1571] 10.222.0.16 () {54 vars in 962 bytes} [Mon Sep 5 12:18:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:18:41.367581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd9650-1a3f-41e3-a970-a2212c87442a None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:41.391128 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 793/1572] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:18:41 2022] POST /identity/v3/users => generated 290 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:18:41.391349 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd9650-1a3f-41e3-a970-a2212c87442a None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:41.393472 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97009ec3-64ef-4f93-87f9-9f80ab4bd01c None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:41.393984 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97009ec3-64ef-4f93-87f9-9f80ab4bd01c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:41.394445 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97009ec3-64ef-4f93-87f9-9f80ab4bd01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:41.400123 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c89c91-bab0-4d75-becf-f7ec6235353a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:41.401145 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c89c91-bab0-4d75-becf-f7ec6235353a admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:41.401613 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c89c91-bab0-4d75-becf-f7ec6235353a admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:41.402121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c89c91-bab0-4d75-becf-f7ec6235353a admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:41.403178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c89c91-bab0-4d75-becf-f7ec6235353a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:41.406936 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c89c91-bab0-4d75-becf-f7ec6235353a admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:41.415848 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-97009ec3-64ef-4f93-87f9-9f80ab4bd01c None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:41.417087 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-97009ec3-64ef-4f93-87f9-9f80ab4bd01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:41.426401 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 794/1573] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:41.427593 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15bfce-9290-4bd0-a78b-90daa2bbdb89 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:41.428257 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15bfce-9290-4bd0-a78b-90daa2bbdb89 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:41.428257 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15bfce-9290-4bd0-a78b-90daa2bbdb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:41.448928 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 780/1574] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:18:41 2022] POST /identity/v3/users => generated 290 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:18:41.450261 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-9d15bfce-9290-4bd0-a78b-90daa2bbdb89 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:41.450951 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f92024-4ae5-4223-8337-032e3edf868f None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:41.451335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f92024-4ae5-4223-8337-032e3edf868f None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:41.451646 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f92024-4ae5-4223-8337-032e3edf868f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:41.451845 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9d15bfce-9290-4bd0-a78b-90daa2bbdb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:41.461676 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 795/1575] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:41.463213 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15bfce-9290-4bd0-a78b-90daa2bbdb89 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:41.478992 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-e4f92024-4ae5-4223-8337-032e3edf868f None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:41.481017 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-e4f92024-4ae5-4223-8337-032e3edf868f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:41.493376 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 781/1576] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:41.494261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f92024-4ae5-4223-8337-032e3edf868f None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:41.505424 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f92024-4ae5-4223-8337-032e3edf868f None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:41.513982 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c62ae9b-1cd3-4d4b-b84e-31d4aab71c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:41.515035 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62ae9b-1cd3-4d4b-b84e-31d4aab71c72 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:41.515850 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62ae9b-1cd3-4d4b-b84e-31d4aab71c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:41.516694 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62ae9b-1cd3-4d4b-b84e-31d4aab71c72 admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:41.517479 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c62ae9b-1cd3-4d4b-b84e-31d4aab71c72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:18:41.518005 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c62ae9b-1cd3-4d4b-b84e-31d4aab71c72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:41.520847 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c62ae9b-1cd3-4d4b-b84e-31d4aab71c72 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:41.531090 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15bfce-9290-4bd0-a78b-90daa2bbdb89 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:41.535966 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03717a29-52bf-4dc1-95e8-2845b763c7ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:41.536723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03717a29-52bf-4dc1-95e8-2845b763c7ec service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:41.537019 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03717a29-52bf-4dc1-95e8-2845b763c7ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:41.537313 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03717a29-52bf-4dc1-95e8-2845b763c7ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:41.541043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 782/1577] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:18:41.548124 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03717a29-52bf-4dc1-95e8-2845b763c7ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:41.551153 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03717a29-52bf-4dc1-95e8-2845b763c7ec service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:41.551307 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc47aff-800d-4a13-8a7f-315e98af11b2 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:41.551307 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc47aff-800d-4a13-8a7f-315e98af11b2 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:41.551307 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc47aff-800d-4a13-8a7f-315e98af11b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:41.566004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 796/1578] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:18:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:18:41.567879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03717a29-52bf-4dc1-95e8-2845b763c7ec service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:41.587093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03717a29-52bf-4dc1-95e8-2845b763c7ec service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:41.596076 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69939527-9258-4bf4-819f-a9c49d9a6971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:41.597119 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69939527-9258-4bf4-819f-a9c49d9a6971 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:41.597599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69939527-9258-4bf4-819f-a9c49d9a6971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:41.598071 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69939527-9258-4bf4-819f-a9c49d9a6971 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:41.599641 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69939527-9258-4bf4-819f-a9c49d9a6971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:41.602544 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69939527-9258-4bf4-819f-a9c49d9a6971 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:41.617917 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 797/1579] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:18:41.620648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69939527-9258-4bf4-819f-a9c49d9a6971 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:41.625513 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-dfc47aff-800d-4a13-8a7f-315e98af11b2 None None] MFA Rules not processed for user `7cda290418504f0db85d665479a05353`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:41.626838 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-dfc47aff-800d-4a13-8a7f-315e98af11b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:41.632308 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69939527-9258-4bf4-819f-a9c49d9a6971 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:41.637640 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e06c6-24fb-48aa-bf16-910ff32d4a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:41.638239 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e06c6-24fb-48aa-bf16-910ff32d4a07 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:41.638387 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e06c6-24fb-48aa-bf16-910ff32d4a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:41.638700 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e06c6-24fb-48aa-bf16-910ff32d4a07 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:41.639542 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e06c6-24fb-48aa-bf16-910ff32d4a07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:18:41.640048 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e06c6-24fb-48aa-bf16-910ff32d4a07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:41.645471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e06c6-24fb-48aa-bf16-910ff32d4a07 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:41.667991 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 798/1580] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:18:41.668924 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e06c6-24fb-48aa-bf16-910ff32d4a07 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:41.686301 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e06c6-24fb-48aa-bf16-910ff32d4a07 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:41.693378 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78078fa1-a973-4e35-86c9-9c5c4d120195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:41.694006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78078fa1-a973-4e35-86c9-9c5c4d120195 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:41.694298 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78078fa1-a973-4e35-86c9-9c5c4d120195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:41.694584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78078fa1-a973-4e35-86c9-9c5c4d120195 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:41.695224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78078fa1-a973-4e35-86c9-9c5c4d120195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:41.696935 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78078fa1-a973-4e35-86c9-9c5c4d120195 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:41.711959 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 799/1581] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:18:41.713404 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78078fa1-a973-4e35-86c9-9c5c4d120195 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:41.720359 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:41.729310 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78078fa1-a973-4e35-86c9-9c5c4d120195 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:41.738395 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce51333-f965-4aaf-bfc2-2b0eb60e6a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:41.739068 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce51333-f965-4aaf-bfc2-2b0eb60e6a1a admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:41.739726 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce51333-f965-4aaf-bfc2-2b0eb60e6a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:41.740034 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce51333-f965-4aaf-bfc2-2b0eb60e6a1a admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:41.740720 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce51333-f965-4aaf-bfc2-2b0eb60e6a1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:18:41.741027 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce51333-f965-4aaf-bfc2-2b0eb60e6a1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:41.743205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce51333-f965-4aaf-bfc2-2b0eb60e6a1a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:41.760374 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 800/1582] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:18:41.763866 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce51333-f965-4aaf-bfc2-2b0eb60e6a1a admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:41.775935 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 783/1583] 10.222.0.16 () {58 vars in 1059 bytes} [Mon Sep 5 12:18:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:41.776526 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc47aff-800d-4a13-8a7f-315e98af11b2 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:41.777247 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce51333-f965-4aaf-bfc2-2b0eb60e6a1a admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:41.798262 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6cef1-b84f-4355-89e2-18718a79dd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:41.800976 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6cef1-b84f-4355-89e2-18718a79dd44 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:41.802805 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6cef1-b84f-4355-89e2-18718a79dd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:41.803262 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6cef1-b84f-4355-89e2-18718a79dd44 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:41.803732 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc47aff-800d-4a13-8a7f-315e98af11b2 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:41.804454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc6cef1-b84f-4355-89e2-18718a79dd44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:18:41.805701 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc6cef1-b84f-4355-89e2-18718a79dd44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:41.807844 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc6cef1-b84f-4355-89e2-18718a79dd44 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:41.813802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c316d289-be64-4f2e-a6ea-16494351b425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:41.815263 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316d289-be64-4f2e-a6ea-16494351b425 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:41.815690 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316d289-be64-4f2e-a6ea-16494351b425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:41.816068 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316d289-be64-4f2e-a6ea-16494351b425 admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:41.816987 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c316d289-be64-4f2e-a6ea-16494351b425 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:41.820060 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c316d289-be64-4f2e-a6ea-16494351b425 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:41.826704 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 801/1584] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:18:41.828292 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6cef1-b84f-4355-89e2-18718a79dd44 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:41.846962 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6cef1-b84f-4355-89e2-18718a79dd44 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:41.853805 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6663f1-8be0-4e8b-a668-e1dd6529f63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:41.854857 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6663f1-8be0-4e8b-a668-e1dd6529f63b admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:41.855117 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6663f1-8be0-4e8b-a668-e1dd6529f63b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:41.855487 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6663f1-8be0-4e8b-a668-e1dd6529f63b admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:41.856167 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6663f1-8be0-4e8b-a668-e1dd6529f63b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:41.857855 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6663f1-8be0-4e8b-a668-e1dd6529f63b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:41.859126 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 784/1585] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:18:41 2022] POST /identity/v3/projects => generated 403 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:18:41.859817 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c316d289-be64-4f2e-a6ea-16494351b425 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:41.877189 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 802/1586] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:18:41.879012 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6663f1-8be0-4e8b-a668-e1dd6529f63b admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:41.894562 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6663f1-8be0-4e8b-a668-e1dd6529f63b admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:41.915099 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:41.958128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c316d289-be64-4f2e-a6ea-16494351b425 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:41.964645 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f82366-807e-4b0f-b13e-cba0ed507e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:41.964645 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f82366-807e-4b0f-b13e-cba0ed507e28 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:41.965041 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f82366-807e-4b0f-b13e-cba0ed507e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:41.965181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f82366-807e-4b0f-b13e-cba0ed507e28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:41.966019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f82366-807e-4b0f-b13e-cba0ed507e28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:41.970612 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f82366-807e-4b0f-b13e-cba0ed507e28 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:41.997235 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f5682-529d-4062-a492-86c52d020f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:41.997885 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f5682-529d-4062-a492-86c52d020f75 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:41.998856 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f5682-529d-4062-a492-86c52d020f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:41.999348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f5682-529d-4062-a492-86c52d020f75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:42.000142 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843f5682-529d-4062-a492-86c52d020f75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:42.002477 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843f5682-529d-4062-a492-86c52d020f75 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:42.024776 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 803/1587] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:18:42.024776 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f5682-529d-4062-a492-86c52d020f75 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:42.027638 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 785/1588] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:18:41 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:18:42.029087 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f82366-807e-4b0f-b13e-cba0ed507e28 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:42.042388 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f5682-529d-4062-a492-86c52d020f75 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:42.133298 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f82366-807e-4b0f-b13e-cba0ed507e28 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:42.140312 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce8ddb-c2dd-4d99-844b-7a82acbb93b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:42.140964 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcba308-1288-45b5-b8e5-66cab95360b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:42.141102 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce8ddb-c2dd-4d99-844b-7a82acbb93b4 service nova] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:42.141400 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce8ddb-c2dd-4d99-844b-7a82acbb93b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:42.141642 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce8ddb-c2dd-4d99-844b-7a82acbb93b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:42.141811 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcba308-1288-45b5-b8e5-66cab95360b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:42.142128 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcba308-1288-45b5-b8e5-66cab95360b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:42.143207 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcba308-1288-45b5-b8e5-66cab95360b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:42.143934 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcba308-1288-45b5-b8e5-66cab95360b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:42.146312 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcba308-1288-45b5-b8e5-66cab95360b2 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:42.195754 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 804/1589] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:18:42 2022] POST /identity/v3/projects => generated 375 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:18:42.198320 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcba308-1288-45b5-b8e5-66cab95360b2 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:42.237164 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ce8ddb-c2dd-4d99-844b-7a82acbb93b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:42.240596 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ce8ddb-c2dd-4d99-844b-7a82acbb93b4 service nova] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:42.259588 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 786/1590] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:18:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:18:42.261256 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce8ddb-c2dd-4d99-844b-7a82acbb93b4 service nova] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:42.279594 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce8ddb-c2dd-4d99-844b-7a82acbb93b4 service nova] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:42.302446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:42.323259 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcba308-1288-45b5-b8e5-66cab95360b2 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:42.330685 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca1ed6-9a26-4e1b-a955-cd5ed2cc4d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:42.331283 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca1ed6-9a26-4e1b-a955-cd5ed2cc4d95 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:42.332256 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca1ed6-9a26-4e1b-a955-cd5ed2cc4d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:42.332560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca1ed6-9a26-4e1b-a955-cd5ed2cc4d95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:42.333206 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fca1ed6-9a26-4e1b-a955-cd5ed2cc4d95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:42.335715 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fca1ed6-9a26-4e1b-a955-cd5ed2cc4d95 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:42.376258 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 805/1591] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:18:42 2022] POST /identity/v3/projects => generated 365 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:18:42.377717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca1ed6-9a26-4e1b-a955-cd5ed2cc4d95 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:42.398628 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:42.413624 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa68b9-1a06-4774-8733-fa8eac025cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:42.414432 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa68b9-1a06-4774-8733-fa8eac025cd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:42.414742 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa68b9-1a06-4774-8733-fa8eac025cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:42.415016 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa68b9-1a06-4774-8733-fa8eac025cd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:42.416043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defa68b9-1a06-4774-8733-fa8eac025cd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:42.418857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defa68b9-1a06-4774-8733-fa8eac025cd0 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:42.463319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca1ed6-9a26-4e1b-a955-cd5ed2cc4d95 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:42.469190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d11b4d-2e18-4e33-9588-bb638c28205b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:42.470210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d11b4d-2e18-4e33-9588-bb638c28205b admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:42.470428 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d11b4d-2e18-4e33-9588-bb638c28205b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:42.470814 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d11b4d-2e18-4e33-9588-bb638c28205b admin admin] PATH_INFO: `/v3/projects/82831792f11f4dcd9770f009448e796e/users/34b23635a4a443eea96fb7cf0f7bc5d5/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:42.481109 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 787/1592] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:18:42 2022] POST /identity/v3/users => generated 302 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:18:42.481109 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa68b9-1a06-4774-8733-fa8eac025cd0 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:42.492590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d11b4d-2e18-4e33-9588-bb638c28205b admin admin] RBAC: Authorizing `identity:create_grant(project_id=82831792f11f4dcd9770f009448e796e, user_id=34b23635a4a443eea96fb7cf0f7bc5d5, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:42.496328 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d11b4d-2e18-4e33-9588-bb638c28205b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:42.510121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa68b9-1a06-4774-8733-fa8eac025cd0 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:42.525426 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 806/1593] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:18:42 2022] PUT /identity/v3/projects/82831792f11f4dcd9770f009448e796e/users/34b23635a4a443eea96fb7cf0f7bc5d5/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:18:42.526563 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d11b4d-2e18-4e33-9588-bb638c28205b admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:42.619517 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf139f9f-b69a-4804-a133-c578a7ed1543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:42.620309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf139f9f-b69a-4804-a133-c578a7ed1543 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:42.620592 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf139f9f-b69a-4804-a133-c578a7ed1543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:42.620921 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf139f9f-b69a-4804-a133-c578a7ed1543 admin admin] PATH_INFO: `/v3/projects/09ba610aa04549c3ab792787257f4372/users/6fb28ddd5511425a93c7b486d78a99eb/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:42.641142 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf139f9f-b69a-4804-a133-c578a7ed1543 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09ba610aa04549c3ab792787257f4372, user_id=6fb28ddd5511425a93c7b486d78a99eb, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:42.644104 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf139f9f-b69a-4804-a133-c578a7ed1543 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:42.648003 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d11b4d-2e18-4e33-9588-bb638c28205b admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:42.659312 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a1f2e0-1627-4586-8eef-0fb0ad12bac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:42.660775 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1f2e0-1627-4586-8eef-0fb0ad12bac4 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:42.661400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1f2e0-1627-4586-8eef-0fb0ad12bac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:42.661886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1f2e0-1627-4586-8eef-0fb0ad12bac4 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:42.663120 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a1f2e0-1627-4586-8eef-0fb0ad12bac4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:42.663915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 788/1594] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:18:42 2022] PUT /identity/v3/projects/09ba610aa04549c3ab792787257f4372/users/6fb28ddd5511425a93c7b486d78a99eb/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:18:42.665070 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf139f9f-b69a-4804-a133-c578a7ed1543 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:42.669820 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a1f2e0-1627-4586-8eef-0fb0ad12bac4 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:42.702672 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 807/1595] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:18:42 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:18:42.703726 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a1f2e0-1627-4586-8eef-0fb0ad12bac4 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:42.722299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a1f2e0-1627-4586-8eef-0fb0ad12bac4 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:42.766101 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf139f9f-b69a-4804-a133-c578a7ed1543 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:42.771939 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa773639-d7c6-45fb-ac5b-585fa8f08135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:42.772903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa773639-d7c6-45fb-ac5b-585fa8f08135 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:42.773388 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa773639-d7c6-45fb-ac5b-585fa8f08135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:42.773839 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa773639-d7c6-45fb-ac5b-585fa8f08135 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:42.775057 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa773639-d7c6-45fb-ac5b-585fa8f08135 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:42.777746 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa773639-d7c6-45fb-ac5b-585fa8f08135 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:42.819132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:42.823017 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 789/1596] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:18:42 2022] POST /identity/v3/users => generated 288 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:18:42.824163 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa773639-d7c6-45fb-ac5b-585fa8f08135 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:42.839463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa773639-d7c6-45fb-ac5b-585fa8f08135 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:42.844590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869ef37-2679-4198-9ccf-81cd9296f958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:42.846343 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869ef37-2679-4198-9ccf-81cd9296f958 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:42.847203 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869ef37-2679-4198-9ccf-81cd9296f958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:42.847482 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869ef37-2679-4198-9ccf-81cd9296f958 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:42.848180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869ef37-2679-4198-9ccf-81cd9296f958 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:42.850147 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869ef37-2679-4198-9ccf-81cd9296f958 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:42.854136 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d9d84-ca11-4a9b-bf25-018fb26e5341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:42.855168 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d9d84-ca11-4a9b-bf25-018fb26e5341 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:42.855704 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d9d84-ca11-4a9b-bf25-018fb26e5341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:42.855979 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d9d84-ca11-4a9b-bf25-018fb26e5341 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:42.856703 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9d9d84-ca11-4a9b-bf25-018fb26e5341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:42.858418 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9d9d84-ca11-4a9b-bf25-018fb26e5341 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:42.878407 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 790/1597] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:18:42.878407 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d5f66-9bcd-43b4-9b3c-8076e98d2623 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:42.878407 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d5f66-9bcd-43b4-9b3c-8076e98d2623 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:42.878407 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d5f66-9bcd-43b4-9b3c-8076e98d2623 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:42.879299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 791/1598] 10.222.0.16 () {54 vars in 912 bytes} [Mon Sep 5 12:18:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:18:42.880776 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5be3f-f2ed-4438-8020-fada80cb5071 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:42.881135 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5be3f-f2ed-4438-8020-fada80cb5071 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:42.881403 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5be3f-f2ed-4438-8020-fada80cb5071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:42.889203 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 808/1599] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:18:42 2022] POST /identity/v3/users => generated 283 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:18:42.890618 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac7890-20cb-4b09-a5ff-cd01bfa10f19 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:42.890910 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac7890-20cb-4b09-a5ff-cd01bfa10f19 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:42.891188 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac7890-20cb-4b09-a5ff-cd01bfa10f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:42.919208 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-a0ac7890-20cb-4b09-a5ff-cd01bfa10f19 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:42.919933 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-bbf5be3f-f2ed-4438-8020-fada80cb5071 None None] MFA Rules not processed for user `34b23635a4a443eea96fb7cf0f7bc5d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:42.920579 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a0ac7890-20cb-4b09-a5ff-cd01bfa10f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:42.921390 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-bbf5be3f-f2ed-4438-8020-fada80cb5071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:42.937349 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 809/1600] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:42.940067 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac7890-20cb-4b09-a5ff-cd01bfa10f19 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:42.953823 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac7890-20cb-4b09-a5ff-cd01bfa10f19 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:42.962100 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827df803-f82b-4ca9-ba56-903f2c548926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:42.962803 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827df803-f82b-4ca9-ba56-903f2c548926 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:42.963087 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827df803-f82b-4ca9-ba56-903f2c548926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:42.963913 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827df803-f82b-4ca9-ba56-903f2c548926 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:42.964625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827df803-f82b-4ca9-ba56-903f2c548926 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:42.966757 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827df803-f82b-4ca9-ba56-903f2c548926 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:42.973244 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:42.984340 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 810/1601] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:18:42.985321 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827df803-f82b-4ca9-ba56-903f2c548926 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:42.998742 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827df803-f82b-4ca9-ba56-903f2c548926 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:43.004731 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f99e765-bc49-4ad5-b09f-98a7e00cf38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:43.005370 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f99e765-bc49-4ad5-b09f-98a7e00cf38e admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:43.005653 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f99e765-bc49-4ad5-b09f-98a7e00cf38e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:43.005929 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f99e765-bc49-4ad5-b09f-98a7e00cf38e admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:43.006643 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f99e765-bc49-4ad5-b09f-98a7e00cf38e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:43.009074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f99e765-bc49-4ad5-b09f-98a7e00cf38e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:43.025837 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 811/1602] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:18:43.026855 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f99e765-bc49-4ad5-b09f-98a7e00cf38e admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:43.030513 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 792/1603] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:43.034895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb973197-1a55-4378-baca-cc9f70dce53b None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:43.035183 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb973197-1a55-4378-baca-cc9f70dce53b None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:43.035456 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb973197-1a55-4378-baca-cc9f70dce53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:43.039920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f99e765-bc49-4ad5-b09f-98a7e00cf38e admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:43.045496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86506d-4b42-445f-8e41-cff34ae529cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:43.046440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86506d-4b42-445f-8e41-cff34ae529cc admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:43.046889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86506d-4b42-445f-8e41-cff34ae529cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:43.047394 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86506d-4b42-445f-8e41-cff34ae529cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:43.048267 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86506d-4b42-445f-8e41-cff34ae529cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:43.050740 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86506d-4b42-445f-8e41-cff34ae529cc admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:43.060389 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-bb973197-1a55-4378-baca-cc9f70dce53b None None] MFA Rules not processed for user `7cda290418504f0db85d665479a05353`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:43.063030 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-bb973197-1a55-4378-baca-cc9f70dce53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:43.063980 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 812/1604] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:18:43.065583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86506d-4b42-445f-8e41-cff34ae529cc admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:43.081437 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86506d-4b42-445f-8e41-cff34ae529cc admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:43.087820 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ee4a2-be67-47d1-9fde-b5b4cbab9891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:43.088560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ee4a2-be67-47d1-9fde-b5b4cbab9891 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:43.088914 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ee4a2-be67-47d1-9fde-b5b4cbab9891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:43.089347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ee4a2-be67-47d1-9fde-b5b4cbab9891 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:43.090094 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ee4a2-be67-47d1-9fde-b5b4cbab9891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:43.092666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ee4a2-be67-47d1-9fde-b5b4cbab9891 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:43.106896 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 813/1605] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:18:43.108160 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ee4a2-be67-47d1-9fde-b5b4cbab9891 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:43.161901 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 793/1606] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Sep 5 12:18:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:43.163025 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb973197-1a55-4378-baca-cc9f70dce53b None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:43.176075 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb973197-1a55-4378-baca-cc9f70dce53b None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:43.182201 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e33caa-dbb5-49ea-a86e-2a8ed3365d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:43.182847 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e33caa-dbb5-49ea-a86e-2a8ed3365d21 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:43.183128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e33caa-dbb5-49ea-a86e-2a8ed3365d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:43.183405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e33caa-dbb5-49ea-a86e-2a8ed3365d21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:43.184722 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e33caa-dbb5-49ea-a86e-2a8ed3365d21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:43.185991 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e33caa-dbb5-49ea-a86e-2a8ed3365d21 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:43.206411 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 794/1607] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:18:43.207600 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e33caa-dbb5-49ea-a86e-2a8ed3365d21 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:43.217114 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ee4a2-be67-47d1-9fde-b5b4cbab9891 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:43.223614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c523a553-3cfd-495b-bc78-f41742949115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:43.224479 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523a553-3cfd-495b-bc78-f41742949115 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:43.225548 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523a553-3cfd-495b-bc78-f41742949115 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:43.225930 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523a553-3cfd-495b-bc78-f41742949115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:43.231065 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e33caa-dbb5-49ea-a86e-2a8ed3365d21 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:43.241930 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac27d4c-dc2a-4c7d-935b-724c3b7273ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:43.246991 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c523a553-3cfd-495b-bc78-f41742949115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:43.246991 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c523a553-3cfd-495b-bc78-f41742949115 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:43.248336 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac27d4c-dc2a-4c7d-935b-724c3b7273ad admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:43.248823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac27d4c-dc2a-4c7d-935b-724c3b7273ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:43.249322 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac27d4c-dc2a-4c7d-935b-724c3b7273ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:43.250224 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac27d4c-dc2a-4c7d-935b-724c3b7273ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:43.253032 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac27d4c-dc2a-4c7d-935b-724c3b7273ad admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:43.258357 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 814/1608] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:18:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:18:43.259435 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c523a553-3cfd-495b-bc78-f41742949115 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:43.268974 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 795/1609] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:18:43.270193 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac27d4c-dc2a-4c7d-935b-724c3b7273ad admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:43.270796 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c523a553-3cfd-495b-bc78-f41742949115 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:43.276343 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2a6af-2b40-4a16-912a-bc31be632104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:43.277193 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2a6af-2b40-4a16-912a-bc31be632104 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:43.277600 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2a6af-2b40-4a16-912a-bc31be632104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:43.278033 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2a6af-2b40-4a16-912a-bc31be632104 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:43.278771 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2a6af-2b40-4a16-912a-bc31be632104 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:43.281144 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2a6af-2b40-4a16-912a-bc31be632104 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:43.282534 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac27d4c-dc2a-4c7d-935b-724c3b7273ad admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:43.288686 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30bcc57-6d42-438f-b03c-fa38eb197c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:43.289862 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30bcc57-6d42-438f-b03c-fa38eb197c60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:43.290386 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30bcc57-6d42-438f-b03c-fa38eb197c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:43.290878 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30bcc57-6d42-438f-b03c-fa38eb197c60 admin admin] PATH_INFO: `/v3/projects/e9c33ce92ed7449b9d52c79b9ab24043/users/9aad16aba3da42189a3e1fc8fb0ff1ad/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:43.310860 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30bcc57-6d42-438f-b03c-fa38eb197c60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9c33ce92ed7449b9d52c79b9ab24043, user_id=9aad16aba3da42189a3e1fc8fb0ff1ad, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:43.315730 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30bcc57-6d42-438f-b03c-fa38eb197c60 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:43.324946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 815/1610] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:18:43 2022] POST /identity/v3/projects => generated 381 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:18:43.326319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2a6af-2b40-4a16-912a-bc31be632104 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:43.355575 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 796/1611] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:18:43 2022] PUT /identity/v3/projects/e9c33ce92ed7449b9d52c79b9ab24043/users/9aad16aba3da42189a3e1fc8fb0ff1ad/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:18:43.355575 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30bcc57-6d42-438f-b03c-fa38eb197c60 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:43.426135 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30bcc57-6d42-438f-b03c-fa38eb197c60 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:43.428764 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2a6af-2b40-4a16-912a-bc31be632104 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:43.429908 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6c7ca-4832-4847-a830-15832bb79144 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:43.431644 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6c7ca-4832-4847-a830-15832bb79144 service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:43.432047 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6c7ca-4832-4847-a830-15832bb79144 service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:43.432543 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6c7ca-4832-4847-a830-15832bb79144 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:43.454644 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:43.479586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:43.547046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f6c7ca-4832-4847-a830-15832bb79144 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:43.549563 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f6c7ca-4832-4847-a830-15832bb79144 service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:43.554451 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f56131-102c-4e26-b253-66086bfe865f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:43.555620 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f56131-102c-4e26-b253-66086bfe865f admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:43.556397 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f56131-102c-4e26-b253-66086bfe865f admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:43.557138 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f56131-102c-4e26-b253-66086bfe865f admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:43.558446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f56131-102c-4e26-b253-66086bfe865f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:43.561836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f56131-102c-4e26-b253-66086bfe865f admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:43.569637 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 797/1612] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:18:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:18:43.570502 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6c7ca-4832-4847-a830-15832bb79144 service cinder] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:43.586784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6c7ca-4832-4847-a830-15832bb79144 service cinder] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:43.595227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8aeac-8f37-444f-ac35-312b872c294f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:43.597730 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8aeac-8f37-444f-ac35-312b872c294f admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:43.598403 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8aeac-8f37-444f-ac35-312b872c294f admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:43.599127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8aeac-8f37-444f-ac35-312b872c294f admin admin] PATH_INFO: `/v3/projects/09d82afaf78a497395c3b4ba0b8294fd/users/02de21bffe154077866824e2033608fc/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:43.615989 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 816/1613] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:18:43.619084 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f56131-102c-4e26-b253-66086bfe865f admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:43.635391 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de8aeac-8f37-444f-ac35-312b872c294f admin admin] RBAC: Authorizing `identity:create_grant(project_id=09d82afaf78a497395c3b4ba0b8294fd, user_id=02de21bffe154077866824e2033608fc, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:43.637119 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f56131-102c-4e26-b253-66086bfe865f admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:43.644284 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da177f-f8cf-4355-8362-d61d7af0c7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:43.645959 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da177f-f8cf-4355-8362-d61d7af0c7a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:43.646189 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da177f-f8cf-4355-8362-d61d7af0c7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:43.646550 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da177f-f8cf-4355-8362-d61d7af0c7a3 admin admin] PATH_INFO: `/v3/projects/8f21d577dc6d44a7b05ee0a0232bed6c/users/45bd61574c6c4ebb9d6fe0c87228065f/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:43.647670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de8aeac-8f37-444f-ac35-312b872c294f admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:43.670574 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4da177f-f8cf-4355-8362-d61d7af0c7a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f21d577dc6d44a7b05ee0a0232bed6c, user_id=45bd61574c6c4ebb9d6fe0c87228065f, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:43.674350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4da177f-f8cf-4355-8362-d61d7af0c7a3 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:43.683851 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 798/1614] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:18:43 2022] PUT /identity/v3/projects/09d82afaf78a497395c3b4ba0b8294fd/users/02de21bffe154077866824e2033608fc/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:18:43.701457 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8aeac-8f37-444f-ac35-312b872c294f admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:43.709743 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 817/1615] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:18:43 2022] PUT /identity/v3/projects/8f21d577dc6d44a7b05ee0a0232bed6c/users/45bd61574c6c4ebb9d6fe0c87228065f/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:18:43.714048 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09903ca-f6c7-4961-a4a5-cf8aa39af857 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:43.714392 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09903ca-f6c7-4961-a4a5-cf8aa39af857 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:43.714601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09903ca-f6c7-4961-a4a5-cf8aa39af857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:43.795399 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-b09903ca-f6c7-4961-a4a5-cf8aa39af857 None None] MFA Rules not processed for user `9aad16aba3da42189a3e1fc8fb0ff1ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:43.798348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b09903ca-f6c7-4961-a4a5-cf8aa39af857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:43.888247 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8aeac-8f37-444f-ac35-312b872c294f admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:43.954037 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 818/1616] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:43 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:43.955128 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09903ca-f6c7-4961-a4a5-cf8aa39af857 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:44.005171 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7d7a5f-1c64-42a0-8e38-1976067b5dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:44.006755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d7a5f-1c64-42a0-8e38-1976067b5dcb admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:44.007727 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d7a5f-1c64-42a0-8e38-1976067b5dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:44.008174 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d7a5f-1c64-42a0-8e38-1976067b5dcb admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:44.009311 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7d7a5f-1c64-42a0-8e38-1976067b5dcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:44.016090 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7d7a5f-1c64-42a0-8e38-1976067b5dcb admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:44.148042 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 799/1617] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:18:43 2022] POST /identity/v3/users => generated 297 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:18:44.148042 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f339b-de7d-43eb-910c-fecedb3c29c3 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:44.148042 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f339b-de7d-43eb-910c-fecedb3c29c3 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:44.148042 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f339b-de7d-43eb-910c-fecedb3c29c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:44.151951 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09903ca-f6c7-4961-a4a5-cf8aa39af857 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:44.168101 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03962518-e5b9-491c-9bd1-449e0bedcbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:44.171230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03962518-e5b9-491c-9bd1-449e0bedcbed admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:44.171859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03962518-e5b9-491c-9bd1-449e0bedcbed admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:44.177284 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-f24f339b-de7d-43eb-910c-fecedb3c29c3 None None] MFA Rules not processed for user `02de21bffe154077866824e2033608fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:44.181167 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-f24f339b-de7d-43eb-910c-fecedb3c29c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:44.181312 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03962518-e5b9-491c-9bd1-449e0bedcbed admin admin] PATH_INFO: `/v3/projects/c87b2c6b54de4dcfa8dd11e47e4466d1/users/b5dfea46fd2f4f9e8111117b6852d3c7/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:44.210425 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03962518-e5b9-491c-9bd1-449e0bedcbed admin admin] RBAC: Authorizing `identity:create_grant(project_id=c87b2c6b54de4dcfa8dd11e47e4466d1, user_id=b5dfea46fd2f4f9e8111117b6852d3c7, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:44.214067 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:44.214353 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03962518-e5b9-491c-9bd1-449e0bedcbed admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:44.238842 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:44.252505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 819/1618] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:18:43 2022] PUT /identity/v3/projects/c87b2c6b54de4dcfa8dd11e47e4466d1/users/b5dfea46fd2f4f9e8111117b6852d3c7/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:18:44.254756 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191f382-f8f1-46c7-9081-d3c87bb55794 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:44.255294 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191f382-f8f1-46c7-9081-d3c87bb55794 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:44.259651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191f382-f8f1-46c7-9081-d3c87bb55794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:44.300025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-7191f382-f8f1-46c7-9081-d3c87bb55794 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:44.301459 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7191f382-f8f1-46c7-9081-d3c87bb55794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:44.319175 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 800/1619] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:44.321266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f339b-de7d-43eb-910c-fecedb3c29c3 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:44.346442 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f339b-de7d-43eb-910c-fecedb3c29c3 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:44.398757 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 820/1620] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:44.402801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191f382-f8f1-46c7-9081-d3c87bb55794 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:44.419488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191f382-f8f1-46c7-9081-d3c87bb55794 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:44.425496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da062939-9279-43c7-ad4a-440b8d46bf4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:44.426570 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da062939-9279-43c7-ad4a-440b8d46bf4b admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:44.427062 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da062939-9279-43c7-ad4a-440b8d46bf4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:44.427794 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da062939-9279-43c7-ad4a-440b8d46bf4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:44.428819 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da062939-9279-43c7-ad4a-440b8d46bf4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:44.430977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da062939-9279-43c7-ad4a-440b8d46bf4b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:44.461759 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 821/1621] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:18:44.463458 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9cdd9-43d0-491b-b7b1-31c0d4657087 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:44.463815 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9cdd9-43d0-491b-b7b1-31c0d4657087 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:44.477172 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9cdd9-43d0-491b-b7b1-31c0d4657087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:44.527430 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96acd3-3c7b-406a-a715-46e9f75301d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:44.529461 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96acd3-3c7b-406a-a715-46e9f75301d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:44.530329 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96acd3-3c7b-406a-a715-46e9f75301d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:44.534565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96acd3-3c7b-406a-a715-46e9f75301d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:44.536199 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96acd3-3c7b-406a-a715-46e9f75301d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:44.540432 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96acd3-3c7b-406a-a715-46e9f75301d7 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:44.606883 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-25c9cdd9-43d0-491b-b7b1-31c0d4657087 None None] MFA Rules not processed for user `b5dfea46fd2f4f9e8111117b6852d3c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:44.611193 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-25c9cdd9-43d0-491b-b7b1-31c0d4657087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:44.650795 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 801/1622] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:18:44 2022] POST /identity/v3/projects => generated 405 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:18:44.654252 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5a0ec-903d-4a43-b17a-84f15d56afbd None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:44.654459 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5a0ec-903d-4a43-b17a-84f15d56afbd None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:44.654895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5a0ec-903d-4a43-b17a-84f15d56afbd None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:44.658150 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 802/1623] 10.222.0.16 () {54 vars in 962 bytes} [Mon Sep 5 12:18:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:18:44.659933 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5a0ec-903d-4a43-b17a-84f15d56afbd None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:44.662996 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-74e5a0ec-903d-4a43-b17a-84f15d56afbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:44.789730 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 822/1624] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:44.791314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9cdd9-43d0-491b-b7b1-31c0d4657087 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:44.827069 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9cdd9-43d0-491b-b7b1-31c0d4657087 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:44.837462 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5a0ec-903d-4a43-b17a-84f15d56afbd None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:44.844682 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a45a3-bb59-4f82-81e6-cff837dc26f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:44.846230 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a45a3-bb59-4f82-81e6-cff837dc26f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:44.846771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a45a3-bb59-4f82-81e6-cff837dc26f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:44.847207 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a45a3-bb59-4f82-81e6-cff837dc26f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:44.986667 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a45a3-bb59-4f82-81e6-cff837dc26f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:44.994504 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a45a3-bb59-4f82-81e6-cff837dc26f8 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:45.002989 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:45.007902 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b543c-a284-4c35-b275-021d20f88a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:45.009641 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b543c-a284-4c35-b275-021d20f88a43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:45.011169 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b543c-a284-4c35-b275-021d20f88a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:45.012193 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b543c-a284-4c35-b275-021d20f88a43 admin admin] PATH_INFO: `/v3/projects/7efc6841b58740b8b56fd5b50dccfd19/users/642dc48b0b394b25a15c996af3a1637f/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:45.023880 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 803/1625] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:18:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:18:45.025908 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a45a3-bb59-4f82-81e6-cff837dc26f8 service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:45.057100 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a45a3-bb59-4f82-81e6-cff837dc26f8 service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:45.061105 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b543c-a284-4c35-b275-021d20f88a43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7efc6841b58740b8b56fd5b50dccfd19, user_id=642dc48b0b394b25a15c996af3a1637f, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:45.069338 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d46fc8-372a-490c-935e-d9d3a0a1c55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:45.071452 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d46fc8-372a-490c-935e-d9d3a0a1c55c admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:45.072391 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d46fc8-372a-490c-935e-d9d3a0a1c55c admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:45.073197 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d46fc8-372a-490c-935e-d9d3a0a1c55c admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:45.075362 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d46fc8-372a-490c-935e-d9d3a0a1c55c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:45.076943 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b543c-a284-4c35-b275-021d20f88a43 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:45.082246 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d46fc8-372a-490c-935e-d9d3a0a1c55c admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:45.105100 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:45.119680 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 823/1626] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:18:44 2022] PUT /identity/v3/projects/7efc6841b58740b8b56fd5b50dccfd19/users/642dc48b0b394b25a15c996af3a1637f/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:18:45.123185 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace7a36-c6a4-419f-abc9-abab220e79d7 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:45.132367 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace7a36-c6a4-419f-abc9-abab220e79d7 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:45.133329 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace7a36-c6a4-419f-abc9-abab220e79d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:45.139145 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 804/1627] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:18:45 2022] POST /identity/v3/users => generated 309 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:18:45.140369 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d46fc8-372a-490c-935e-d9d3a0a1c55c admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:45.202214 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-cace7a36-c6a4-419f-abc9-abab220e79d7 None None] MFA Rules not processed for user `7cda290418504f0db85d665479a05353`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:45.205730 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-cace7a36-c6a4-419f-abc9-abab220e79d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:45.308998 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d46fc8-372a-490c-935e-d9d3a0a1c55c admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:45.475871 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 824/1628] 10.222.0.16 () {58 vars in 1059 bytes} [Mon Sep 5 12:18:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:45.478376 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace7a36-c6a4-419f-abc9-abab220e79d7 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:45.503383 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b451efa-b26b-4d20-b09e-60dacb77abe1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:45.504721 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b451efa-b26b-4d20-b09e-60dacb77abe1 service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:45.517771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b451efa-b26b-4d20-b09e-60dacb77abe1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:45.519494 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b451efa-b26b-4d20-b09e-60dacb77abe1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:45.661407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace7a36-c6a4-419f-abc9-abab220e79d7 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:45.679017 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bec4b6-31da-4923-87be-5c06f352f985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:45.679696 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bec4b6-31da-4923-87be-5c06f352f985 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:45.679981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bec4b6-31da-4923-87be-5c06f352f985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:45.680255 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bec4b6-31da-4923-87be-5c06f352f985 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:45.680916 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bec4b6-31da-4923-87be-5c06f352f985 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:45.682566 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bec4b6-31da-4923-87be-5c06f352f985 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:45.712523 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b451efa-b26b-4d20-b09e-60dacb77abe1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:45.727258 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 825/1629] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:18:45.728267 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bec4b6-31da-4923-87be-5c06f352f985 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:45.728846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b451efa-b26b-4d20-b09e-60dacb77abe1 service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:45.748506 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bec4b6-31da-4923-87be-5c06f352f985 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:45.755899 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 805/1630] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:18:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:18:45.757948 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b451efa-b26b-4d20-b09e-60dacb77abe1 service cinder] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:45.772631 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3832081-7acd-4ae2-833b-5e7f85864ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:45.776394 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3832081-7acd-4ae2-833b-5e7f85864ea9 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:45.777586 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3832081-7acd-4ae2-833b-5e7f85864ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:45.777980 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3832081-7acd-4ae2-833b-5e7f85864ea9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:45.779072 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b451efa-b26b-4d20-b09e-60dacb77abe1 service cinder] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:45.779950 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3832081-7acd-4ae2-833b-5e7f85864ea9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:45.784314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3832081-7acd-4ae2-833b-5e7f85864ea9 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:45.792925 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2c522-2fb2-4255-83b9-298d878f6614 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:45.793587 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2c522-2fb2-4255-83b9-298d878f6614 service nova] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:45.797274 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2c522-2fb2-4255-83b9-298d878f6614 service nova] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:45.797586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2c522-2fb2-4255-83b9-298d878f6614 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:45.812448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 826/1631] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:18:45.813663 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3832081-7acd-4ae2-833b-5e7f85864ea9 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:45.826968 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3832081-7acd-4ae2-833b-5e7f85864ea9 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:45.832462 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500ccb2-555e-4ec3-8fc4-1c3ff7388615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:45.833920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500ccb2-555e-4ec3-8fc4-1c3ff7388615 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:45.834597 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500ccb2-555e-4ec3-8fc4-1c3ff7388615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:45.834597 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500ccb2-555e-4ec3-8fc4-1c3ff7388615 admin admin] PATH_INFO: `/v3/projects/7efc6841b58740b8b56fd5b50dccfd19/users/642dc48b0b394b25a15c996af3a1637f/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:45.838370 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4500ccb2-555e-4ec3-8fc4-1c3ff7388615 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7efc6841b58740b8b56fd5b50dccfd19, user_id=642dc48b0b394b25a15c996af3a1637f, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:45.842010 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4500ccb2-555e-4ec3-8fc4-1c3ff7388615 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:45.866662 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 827/1632] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:18:45 2022] PUT /identity/v3/projects/7efc6841b58740b8b56fd5b50dccfd19/users/642dc48b0b394b25a15c996af3a1637f/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:18:45.866662 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500ccb2-555e-4ec3-8fc4-1c3ff7388615 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:45.919011 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:45.932655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd2c522-2fb2-4255-83b9-298d878f6614 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:45.934984 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd2c522-2fb2-4255-83b9-298d878f6614 service nova] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:45.947015 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 806/1633] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:18:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:18:45.948552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1fcb3e-1aeb-48f1-9592-92a84a15401b None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:45.949028 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1fcb3e-1aeb-48f1-9592-92a84a15401b None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:45.949410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1fcb3e-1aeb-48f1-9592-92a84a15401b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:46.001822 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-8e1fcb3e-1aeb-48f1-9592-92a84a15401b None None] MFA Rules not processed for user `642dc48b0b394b25a15c996af3a1637f`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:46.008162 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-8e1fcb3e-1aeb-48f1-9592-92a84a15401b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:46.115763 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500ccb2-555e-4ec3-8fc4-1c3ff7388615 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:46.134289 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c622108-0426-447f-8c57-5104bd20b9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:46.135010 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c622108-0426-447f-8c57-5104bd20b9cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:46.135289 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c622108-0426-447f-8c57-5104bd20b9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:46.135585 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c622108-0426-447f-8c57-5104bd20b9cd admin admin] PATH_INFO: `/v3/projects/5cc27d18198e44d78fd91e743d0ae9f7/users/11702742eec9402a89b006b95c45b89c/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:46.150615 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:46.174324 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c622108-0426-447f-8c57-5104bd20b9cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cc27d18198e44d78fd91e743d0ae9f7, user_id=11702742eec9402a89b006b95c45b89c, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:46.177252 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c622108-0426-447f-8c57-5104bd20b9cd admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:46.217226 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 828/1634] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:18:45 2022] PUT /identity/v3/projects/5cc27d18198e44d78fd91e743d0ae9f7/users/11702742eec9402a89b006b95c45b89c/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:18:46.219041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c622108-0426-447f-8c57-5104bd20b9cd admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:46.386149 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 807/1635] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:45 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 434 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:46.396292 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1fcb3e-1aeb-48f1-9592-92a84a15401b None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:46.421152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c622108-0426-447f-8c57-5104bd20b9cd admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:46.443101 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f0329-d38a-477c-9367-a8cc8b22ac46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:46.443958 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f0329-d38a-477c-9367-a8cc8b22ac46 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:46.444275 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f0329-d38a-477c-9367-a8cc8b22ac46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:46.444554 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f0329-d38a-477c-9367-a8cc8b22ac46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:46.445255 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774f0329-d38a-477c-9367-a8cc8b22ac46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:46.446964 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774f0329-d38a-477c-9367-a8cc8b22ac46 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:46.491719 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 829/1636] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:18:46.492720 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f0329-d38a-477c-9367-a8cc8b22ac46 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:46.515681 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f0329-d38a-477c-9367-a8cc8b22ac46 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:46.527510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32c8199-d80e-433c-9f3d-aaa0bca01b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:46.528234 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c8199-d80e-433c-9f3d-aaa0bca01b80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:46.528521 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c8199-d80e-433c-9f3d-aaa0bca01b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:46.528798 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c8199-d80e-433c-9f3d-aaa0bca01b80 admin admin] PATH_INFO: `/v3/projects/5cc27d18198e44d78fd91e743d0ae9f7/users/11702742eec9402a89b006b95c45b89c/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:46.534998 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32c8199-d80e-433c-9f3d-aaa0bca01b80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cc27d18198e44d78fd91e743d0ae9f7, user_id=11702742eec9402a89b006b95c45b89c, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:46.538485 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32c8199-d80e-433c-9f3d-aaa0bca01b80 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:46.577068 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 830/1637] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:18:46 2022] PUT /identity/v3/projects/5cc27d18198e44d78fd91e743d0ae9f7/users/11702742eec9402a89b006b95c45b89c/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:18:46.578668 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03b91b-0974-42e7-937d-756d3812e09a None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:46.578961 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03b91b-0974-42e7-937d-756d3812e09a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:46.579240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03b91b-0974-42e7-937d-756d3812e09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:46.648131 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1fcb3e-1aeb-48f1-9592-92a84a15401b None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:46.653958 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-7a03b91b-0974-42e7-937d-756d3812e09a None None] MFA Rules not processed for user `11702742eec9402a89b006b95c45b89c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:46.655232 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7a03b91b-0974-42e7-937d-756d3812e09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:46.802175 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3ed6b-fb29-4c1d-8ae2-a250970cc55c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:46.802824 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3ed6b-fb29-4c1d-8ae2-a250970cc55c service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:46.803492 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3ed6b-fb29-4c1d-8ae2-a250970cc55c service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:46.803492 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3ed6b-fb29-4c1d-8ae2-a250970cc55c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:46.858587 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 831/1638] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:46 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:46.863763 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5dfb4b-82ff-4a9c-be37-72c67b369652 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:46.864599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5dfb4b-82ff-4a9c-be37-72c67b369652 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:46.866891 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5dfb4b-82ff-4a9c-be37-72c67b369652 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:46.869324 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 832/1639] 10.222.0.16 () {54 vars in 912 bytes} [Mon Sep 5 12:18:46 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:18:46.877143 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46d6f4-6a11-4eec-bc46-f2cacb74bf54 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:46.877888 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46d6f4-6a11-4eec-bc46-f2cacb74bf54 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:46.878908 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46d6f4-6a11-4eec-bc46-f2cacb74bf54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:46.925706 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:46.932652 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-ec46d6f4-6a11-4eec-bc46-f2cacb74bf54 None None] MFA Rules not processed for user `7cda290418504f0db85d665479a05353`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:46.934617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ec46d6f4-6a11-4eec-bc46-f2cacb74bf54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:46.980390 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3ed6b-fb29-4c1d-8ae2-a250970cc55c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:46.987596 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3ed6b-fb29-4c1d-8ae2-a250970cc55c service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:47.012278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:47.044965 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 833/1640] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Sep 5 12:18:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:18:47.054207 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 808/1641] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:18:46 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 659 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:18:47.102445 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46d6f4-6a11-4eec-bc46-f2cacb74bf54 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:47.113926 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46d6f4-6a11-4eec-bc46-f2cacb74bf54 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:47.119871 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be384582-3f4f-4ca1-8d94-8d31d9d92ac4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:47.120698 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be384582-3f4f-4ca1-8d94-8d31d9d92ac4 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:47.121861 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be384582-3f4f-4ca1-8d94-8d31d9d92ac4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:47.123622 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be384582-3f4f-4ca1-8d94-8d31d9d92ac4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:47.163717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be384582-3f4f-4ca1-8d94-8d31d9d92ac4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:47.166570 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3ed6b-fb29-4c1d-8ae2-a250970cc55c service cinder] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:47.181319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be384582-3f4f-4ca1-8d94-8d31d9d92ac4 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:47.188232 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3ed6b-fb29-4c1d-8ae2-a250970cc55c service cinder] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:47.202685 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c7d0c-4863-46a9-9fea-e23253ea688e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:47.204689 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c7d0c-4863-46a9-9fea-e23253ea688e service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:47.208884 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c7d0c-4863-46a9-9fea-e23253ea688e service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:47.209820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c7d0c-4863-46a9-9fea-e23253ea688e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:47.216795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 834/1642] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:18:47 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:18:47.230910 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129c7d0c-4863-46a9-9fea-e23253ea688e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:47.232731 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129c7d0c-4863-46a9-9fea-e23253ea688e service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:47.251281 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 809/1643] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:18:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:18:47.764311 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be384582-3f4f-4ca1-8d94-8d31d9d92ac4 service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:47.775034 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be384582-3f4f-4ca1-8d94-8d31d9d92ac4 service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:47.792361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b833a078-c7a3-4d8c-8646-1510bba4941e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:47.793283 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833a078-c7a3-4d8c-8646-1510bba4941e service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:47.793789 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833a078-c7a3-4d8c-8646-1510bba4941e service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:47.794142 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833a078-c7a3-4d8c-8646-1510bba4941e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:47.958589 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b833a078-c7a3-4d8c-8646-1510bba4941e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:47.961045 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b833a078-c7a3-4d8c-8646-1510bba4941e service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:47.997316 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 835/1644] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:18:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:18:48.705153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c7d0c-4863-46a9-9fea-e23253ea688e service cinder] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:48.960531 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c7d0c-4863-46a9-9fea-e23253ea688e service cinder] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:48.969564 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c2e1d8-50e1-42c1-ad2f-3b0b65c453a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:48.971508 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c2e1d8-50e1-42c1-ad2f-3b0b65c453a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:48.972296 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c2e1d8-50e1-42c1-ad2f-3b0b65c453a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:48.973276 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c2e1d8-50e1-42c1-ad2f-3b0b65c453a8 admin admin] PATH_INFO: `/v3/users/9aad16aba3da42189a3e1fc8fb0ff1ad` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:48.976163 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c2e1d8-50e1-42c1-ad2f-3b0b65c453a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aad16aba3da42189a3e1fc8fb0ff1ad)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:48.978312 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: WARNING py.warnings [None req-d4c2e1d8-50e1-42c1-ad2f-3b0b65c453a8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 12:18:48.978312 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: warnings.warn(msg) Sep 05 12:18:48.978312 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: Sep 05 12:18:48.989280 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c2e1d8-50e1-42c1-ad2f-3b0b65c453a8 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:49.193989 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-d4c2e1d8-50e1-42c1-ad2f-3b0b65c453a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aad16aba3da42189a3e1fc8fb0ff1ad'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:49.210487 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-d4c2e1d8-50e1-42c1-ad2f-3b0b65c453a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aad16aba3da42189a3e1fc8fb0ff1ad'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:49.212087 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-d4c2e1d8-50e1-42c1-ad2f-3b0b65c453a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aad16aba3da42189a3e1fc8fb0ff1ad'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:49.263700 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-d4c2e1d8-50e1-42c1-ad2f-3b0b65c453a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aad16aba3da42189a3e1fc8fb0ff1ad'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:49.294005 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-d4c2e1d8-50e1-42c1-ad2f-3b0b65c453a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aad16aba3da42189a3e1fc8fb0ff1ad'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:49.310909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 810/1645] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:18:48 2022] DELETE /identity/v3/users/9aad16aba3da42189a3e1fc8fb0ff1ad => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:18:49.316459 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414a2d8-c8b9-4ef2-aa40-0fb723a3ddbf None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:49.317026 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414a2d8-c8b9-4ef2-aa40-0fb723a3ddbf None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:49.317486 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414a2d8-c8b9-4ef2-aa40-0fb723a3ddbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:49.442676 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-8414a2d8-c8b9-4ef2-aa40-0fb723a3ddbf None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:49.452080 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8414a2d8-c8b9-4ef2-aa40-0fb723a3ddbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:49.709138 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 836/1646] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:49.865792 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c2e1d8-50e1-42c1-ad2f-3b0b65c453a8 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:49.867624 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-d4c2e1d8-50e1-42c1-ad2f-3b0b65c453a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:50.047762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:50.065962 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c2e1d8-50e1-42c1-ad2f-3b0b65c453a8 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:50.085276 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000e89d-dfaa-42da-bd82-12fe94f9b81f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:50.085276 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000e89d-dfaa-42da-bd82-12fe94f9b81f service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:50.085276 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000e89d-dfaa-42da-bd82-12fe94f9b81f service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:50.085276 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000e89d-dfaa-42da-bd82-12fe94f9b81f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:50.087099 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8414a2d8-c8b9-4ef2-aa40-0fb723a3ddbf None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:50.088193 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8414a2d8-c8b9-4ef2-aa40-0fb723a3ddbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:50.115330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8414a2d8-c8b9-4ef2-aa40-0fb723a3ddbf None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:50.132599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c4a55-b937-4f76-8e09-335cf51a8705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:50.136148 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9000e89d-dfaa-42da-bd82-12fe94f9b81f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:50.136436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c4a55-b937-4f76-8e09-335cf51a8705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:50.137502 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c4a55-b937-4f76-8e09-335cf51a8705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:50.139090 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c4a55-b937-4f76-8e09-335cf51a8705 admin admin] PATH_INFO: `/v3/users/231eca85785c49628baaee7d6066c3f1` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:50.141381 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9000e89d-dfaa-42da-bd82-12fe94f9b81f service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:50.154482 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c4a55-b937-4f76-8e09-335cf51a8705 admin admin] RBAC: Authorizing `identity:delete_user(user_id=231eca85785c49628baaee7d6066c3f1)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:50.161461 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c4a55-b937-4f76-8e09-335cf51a8705 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:50.180842 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 811/1647] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:18:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:18:50.308103 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-711c4a55-b937-4f76-8e09-335cf51a8705 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '231eca85785c49628baaee7d6066c3f1'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:50.325861 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-711c4a55-b937-4f76-8e09-335cf51a8705 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '231eca85785c49628baaee7d6066c3f1'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:50.328046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-711c4a55-b937-4f76-8e09-335cf51a8705 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '231eca85785c49628baaee7d6066c3f1'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:50.330245 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-711c4a55-b937-4f76-8e09-335cf51a8705 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '231eca85785c49628baaee7d6066c3f1'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:50.352769 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:50.359230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-711c4a55-b937-4f76-8e09-335cf51a8705 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '231eca85785c49628baaee7d6066c3f1'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:50.389536 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 837/1648] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:18:50 2022] DELETE /identity/v3/users/231eca85785c49628baaee7d6066c3f1 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:18:50.395668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5952966-e8e6-4a12-8d1b-122b62ad2b4e None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:50.395963 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5952966-e8e6-4a12-8d1b-122b62ad2b4e None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:50.396245 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5952966-e8e6-4a12-8d1b-122b62ad2b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:50.500748 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-d5952966-e8e6-4a12-8d1b-122b62ad2b4e None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:50.504369 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-d5952966-e8e6-4a12-8d1b-122b62ad2b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:50.748767 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 812/1649] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:50.836274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c4a55-b937-4f76-8e09-335cf51a8705 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:50.838153 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-711c4a55-b937-4f76-8e09-335cf51a8705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:51.137847 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c4a55-b937-4f76-8e09-335cf51a8705 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:51.159307 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be0462-20c2-44bf-8d78-66aa5f28de09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:51.162048 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be0462-20c2-44bf-8d78-66aa5f28de09 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:51.164634 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be0462-20c2-44bf-8d78-66aa5f28de09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:51.165354 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be0462-20c2-44bf-8d78-66aa5f28de09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:51.249384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be0462-20c2-44bf-8d78-66aa5f28de09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:51.253857 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be0462-20c2-44bf-8d78-66aa5f28de09 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:51.298340 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 838/1650] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:18:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:18:51.590085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2c3bd-79df-4fc0-a6ba-3aa0218dbdab None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:51.590953 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2c3bd-79df-4fc0-a6ba-3aa0218dbdab None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:51.590953 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2c3bd-79df-4fc0-a6ba-3aa0218dbdab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:51.655663 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-a0b2c3bd-79df-4fc0-a6ba-3aa0218dbdab None None] MFA Rules not processed for user `83d3b245c185424a9b6e5d69d691f031`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:51.657504 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-a0b2c3bd-79df-4fc0-a6ba-3aa0218dbdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:51.787631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 813/1651] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:51 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:18:51.807541 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be0462-20c2-44bf-8d78-66aa5f28de09 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:51.809797 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-59be0462-20c2-44bf-8d78-66aa5f28de09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:52.044340 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be0462-20c2-44bf-8d78-66aa5f28de09 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:52.064695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42254a-faf5-4a93-9762-a4b1b7b53a0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:52.065335 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42254a-faf5-4a93-9762-a4b1b7b53a0d service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:52.066811 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42254a-faf5-4a93-9762-a4b1b7b53a0d service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:52.067170 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42254a-faf5-4a93-9762-a4b1b7b53a0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:52.092508 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae42254a-faf5-4a93-9762-a4b1b7b53a0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:52.098232 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae42254a-faf5-4a93-9762-a4b1b7b53a0d service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:52.128307 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 839/1652] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:18:51 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:18:52.591249 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2c3bd-79df-4fc0-a6ba-3aa0218dbdab None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:52.593064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-a0b2c3bd-79df-4fc0-a6ba-3aa0218dbdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:52.623838 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2c3bd-79df-4fc0-a6ba-3aa0218dbdab None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:52.637928 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868095f5-1bcb-4ddc-9e69-a27fc55f4563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:52.638597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868095f5-1bcb-4ddc-9e69-a27fc55f4563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:52.638880 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868095f5-1bcb-4ddc-9e69-a27fc55f4563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:52.639143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868095f5-1bcb-4ddc-9e69-a27fc55f4563 admin admin] PATH_INFO: `/v3/projects/e9c33ce92ed7449b9d52c79b9ab24043` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:52.647335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868095f5-1bcb-4ddc-9e69-a27fc55f4563 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9c33ce92ed7449b9d52c79b9ab24043)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:52.655836 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868095f5-1bcb-4ddc-9e69-a27fc55f4563 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:52.918142 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-868095f5-1bcb-4ddc-9e69-a27fc55f4563 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9c33ce92ed7449b9d52c79b9ab24043', 'request_id': 'req-868095f5-1bcb-4ddc-9e69-a27fc55f4563'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:52.935611 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-868095f5-1bcb-4ddc-9e69-a27fc55f4563 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9c33ce92ed7449b9d52c79b9ab24043', 'request_id': 'req-868095f5-1bcb-4ddc-9e69-a27fc55f4563'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:52.949621 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-868095f5-1bcb-4ddc-9e69-a27fc55f4563 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9c33ce92ed7449b9d52c79b9ab24043', 'request_id': 'req-868095f5-1bcb-4ddc-9e69-a27fc55f4563'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:52.983620 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-868095f5-1bcb-4ddc-9e69-a27fc55f4563 admin admin] The token cache is being invalidate because project e9c33ce92ed7449b9d52c79b9ab24043 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:18:52.984284 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-868095f5-1bcb-4ddc-9e69-a27fc55f4563 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:53.000108 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-868095f5-1bcb-4ddc-9e69-a27fc55f4563 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:53.004997 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 814/1653] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:18:52 2022] DELETE /identity/v3/projects/e9c33ce92ed7449b9d52c79b9ab24043 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:18:53.008827 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42254a-faf5-4a93-9762-a4b1b7b53a0d service nova] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:53.036433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42254a-faf5-4a93-9762-a4b1b7b53a0d service nova] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:53.200190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf8289-7829-4db0-8ea4-19bd89a31020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:53.200966 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf8289-7829-4db0-8ea4-19bd89a31020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:53.201287 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf8289-7829-4db0-8ea4-19bd89a31020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:53.201597 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf8289-7829-4db0-8ea4-19bd89a31020 admin admin] PATH_INFO: `/v3/users/11702742eec9402a89b006b95c45b89c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:53.207109 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acf8289-7829-4db0-8ea4-19bd89a31020 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11702742eec9402a89b006b95c45b89c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:53.211902 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acf8289-7829-4db0-8ea4-19bd89a31020 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:53.280850 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:53.366241 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7acf8289-7829-4db0-8ea4-19bd89a31020 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11702742eec9402a89b006b95c45b89c'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:53.389680 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7acf8289-7829-4db0-8ea4-19bd89a31020 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11702742eec9402a89b006b95c45b89c'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:53.390620 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7acf8289-7829-4db0-8ea4-19bd89a31020 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11702742eec9402a89b006b95c45b89c'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:53.393125 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7acf8289-7829-4db0-8ea4-19bd89a31020 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11702742eec9402a89b006b95c45b89c'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:53.439200 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7acf8289-7829-4db0-8ea4-19bd89a31020 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11702742eec9402a89b006b95c45b89c'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:53.471231 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 840/1654] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:18:53 2022] DELETE /identity/v3/users/11702742eec9402a89b006b95c45b89c => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:18:53.900657 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868095f5-1bcb-4ddc-9e69-a27fc55f4563 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:53.903191 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-868095f5-1bcb-4ddc-9e69-a27fc55f4563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:53.937554 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:53.975292 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868095f5-1bcb-4ddc-9e69-a27fc55f4563 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:54.262888 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91080002-2cad-410e-b4a5-1b5f183acc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:54.266718 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91080002-2cad-410e-b4a5-1b5f183acc2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:54.270351 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91080002-2cad-410e-b4a5-1b5f183acc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:54.271383 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91080002-2cad-410e-b4a5-1b5f183acc2c admin admin] PATH_INFO: `/v3/projects/47557a3ca7624ad9b227ea0e51fd414e` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:54.278711 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91080002-2cad-410e-b4a5-1b5f183acc2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=47557a3ca7624ad9b227ea0e51fd414e)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:54.282409 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91080002-2cad-410e-b4a5-1b5f183acc2c admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:54.455767 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-91080002-2cad-410e-b4a5-1b5f183acc2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47557a3ca7624ad9b227ea0e51fd414e', 'request_id': 'req-91080002-2cad-410e-b4a5-1b5f183acc2c'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:54.474407 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-91080002-2cad-410e-b4a5-1b5f183acc2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47557a3ca7624ad9b227ea0e51fd414e', 'request_id': 'req-91080002-2cad-410e-b4a5-1b5f183acc2c'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:54.489083 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-91080002-2cad-410e-b4a5-1b5f183acc2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47557a3ca7624ad9b227ea0e51fd414e', 'request_id': 'req-91080002-2cad-410e-b4a5-1b5f183acc2c'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:54.523738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-91080002-2cad-410e-b4a5-1b5f183acc2c admin admin] The token cache is being invalidate because project 47557a3ca7624ad9b227ea0e51fd414e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:18:54.524351 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-91080002-2cad-410e-b4a5-1b5f183acc2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:54.525356 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-91080002-2cad-410e-b4a5-1b5f183acc2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:54.532559 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 815/1655] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:18:53 2022] DELETE /identity/v3/projects/47557a3ca7624ad9b227ea0e51fd414e => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:18:54.538138 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75309260-e662-4486-9ad0-cf8e239a3379 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:54.539003 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75309260-e662-4486-9ad0-cf8e239a3379 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:54.539003 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75309260-e662-4486-9ad0-cf8e239a3379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:54.587450 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-75309260-e662-4486-9ad0-cf8e239a3379 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:54.588813 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-75309260-e662-4486-9ad0-cf8e239a3379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:54.738141 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 841/1656] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:18:54.750047 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69eb1d-050b-43f0-9e8c-6c41a83d0fd9 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:54.750337 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69eb1d-050b-43f0-9e8c-6c41a83d0fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:54.750591 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69eb1d-050b-43f0-9e8c-6c41a83d0fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:54.826165 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-ce69eb1d-050b-43f0-9e8c-6c41a83d0fd9 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:54.834659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-ce69eb1d-050b-43f0-9e8c-6c41a83d0fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:54.872702 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 816/1657] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:54.874931 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75309260-e662-4486-9ad0-cf8e239a3379 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:54.915733 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75309260-e662-4486-9ad0-cf8e239a3379 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:54.936840 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2ef11-623a-4cd7-9251-3315bf6e488b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:54.937890 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2ef11-623a-4cd7-9251-3315bf6e488b admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:54.938342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2ef11-623a-4cd7-9251-3315bf6e488b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:54.938788 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2ef11-623a-4cd7-9251-3315bf6e488b admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:54.940431 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab2ef11-623a-4cd7-9251-3315bf6e488b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:18:54.940623 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab2ef11-623a-4cd7-9251-3315bf6e488b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:54.943691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab2ef11-623a-4cd7-9251-3315bf6e488b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:54.984146 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 842/1658] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:18:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:18:54.990967 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69eb1d-050b-43f0-9e8c-6c41a83d0fd9 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:55.041262 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69eb1d-050b-43f0-9e8c-6c41a83d0fd9 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:55.054951 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d50b1-1e81-4812-b5b7-006981a7fcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:55.056753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d50b1-1e81-4812-b5b7-006981a7fcfc admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:55.057403 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d50b1-1e81-4812-b5b7-006981a7fcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:55.057963 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d50b1-1e81-4812-b5b7-006981a7fcfc admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:55.059740 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d50b1-1e81-4812-b5b7-006981a7fcfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:55.065847 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d50b1-1e81-4812-b5b7-006981a7fcfc admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:55.139361 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 817/1659] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:18:54 2022] POST /identity/v3/projects => generated 373 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:18:55.146981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2ef11-623a-4cd7-9251-3315bf6e488b admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:55.155049 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d50b1-1e81-4812-b5b7-006981a7fcfc admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:55.160854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-cf8d50b1-1e81-4812-b5b7-006981a7fcfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:55.188287 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2ef11-623a-4cd7-9251-3315bf6e488b admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:55.323211 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d50b1-1e81-4812-b5b7-006981a7fcfc admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:55.331349 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57842f02-f80c-4c8b-87a7-3129def210fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:55.332043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57842f02-f80c-4c8b-87a7-3129def210fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:55.332485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57842f02-f80c-4c8b-87a7-3129def210fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:55.332761 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57842f02-f80c-4c8b-87a7-3129def210fb admin admin] PATH_INFO: `/v3/projects/5cc27d18198e44d78fd91e743d0ae9f7` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:55.334072 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57842f02-f80c-4c8b-87a7-3129def210fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cc27d18198e44d78fd91e743d0ae9f7)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:55.336588 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57842f02-f80c-4c8b-87a7-3129def210fb admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:55.347573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:55.411489 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73063fe0-8297-4133-a65c-35975c850aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:55.412571 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73063fe0-8297-4133-a65c-35975c850aa9 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:55.413091 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73063fe0-8297-4133-a65c-35975c850aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:55.413727 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73063fe0-8297-4133-a65c-35975c850aa9 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:55.414975 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73063fe0-8297-4133-a65c-35975c850aa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:55.422303 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73063fe0-8297-4133-a65c-35975c850aa9 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:55.459976 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-57842f02-f80c-4c8b-87a7-3129def210fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cc27d18198e44d78fd91e743d0ae9f7', 'request_id': 'req-57842f02-f80c-4c8b-87a7-3129def210fb'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:55.481634 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-57842f02-f80c-4c8b-87a7-3129def210fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cc27d18198e44d78fd91e743d0ae9f7', 'request_id': 'req-57842f02-f80c-4c8b-87a7-3129def210fb'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:55.492406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 843/1660] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:18:55 2022] POST /identity/v3/users => generated 287 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:18:55.498295 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-57842f02-f80c-4c8b-87a7-3129def210fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cc27d18198e44d78fd91e743d0ae9f7', 'request_id': 'req-57842f02-f80c-4c8b-87a7-3129def210fb'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:55.499104 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73063fe0-8297-4133-a65c-35975c850aa9 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:55.532379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-57842f02-f80c-4c8b-87a7-3129def210fb admin admin] The token cache is being invalidate because project 5cc27d18198e44d78fd91e743d0ae9f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:18:55.533128 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73063fe0-8297-4133-a65c-35975c850aa9 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:55.533914 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-57842f02-f80c-4c8b-87a7-3129def210fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:55.535844 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-57842f02-f80c-4c8b-87a7-3129def210fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:55.548601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:55.550867 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 818/1661] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:18:55 2022] DELETE /identity/v3/projects/5cc27d18198e44d78fd91e743d0ae9f7 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:18:55.553432 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621ab02-a8bd-41a5-aec5-8b04466b3867 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:55.553918 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621ab02-a8bd-41a5-aec5-8b04466b3867 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:55.554409 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621ab02-a8bd-41a5-aec5-8b04466b3867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:55.632724 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-8621ab02-a8bd-41a5-aec5-8b04466b3867 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:55.634919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-8621ab02-a8bd-41a5-aec5-8b04466b3867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:55.772974 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88a7c8-c5fc-4e73-9fef-8e6162f543e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:55.773662 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88a7c8-c5fc-4e73-9fef-8e6162f543e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:55.774610 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88a7c8-c5fc-4e73-9fef-8e6162f543e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:55.775329 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88a7c8-c5fc-4e73-9fef-8e6162f543e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:55.776754 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88a7c8-c5fc-4e73-9fef-8e6162f543e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:55.782514 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88a7c8-c5fc-4e73-9fef-8e6162f543e5 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:55.828120 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 844/1662] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:18:55.833451 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88a7c8-c5fc-4e73-9fef-8e6162f543e5 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:55.841196 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3d88a7c8-c5fc-4e73-9fef-8e6162f543e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:55.883280 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88a7c8-c5fc-4e73-9fef-8e6162f543e5 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:55.907564 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 819/1663] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:55.907882 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b6844-1b38-41aa-9dc1-be1ad660f0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:55.907882 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b6844-1b38-41aa-9dc1-be1ad660f0aa admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:55.907882 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b6844-1b38-41aa-9dc1-be1ad660f0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:55.907882 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b6844-1b38-41aa-9dc1-be1ad660f0aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:55.908328 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b6844-1b38-41aa-9dc1-be1ad660f0aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:55.909916 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b6844-1b38-41aa-9dc1-be1ad660f0aa admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:55.918729 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396eb9bf-847f-4626-86df-3721c3ab0d38 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:55.920079 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396eb9bf-847f-4626-86df-3721c3ab0d38 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:55.920986 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396eb9bf-847f-4626-86df-3721c3ab0d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:55.963971 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 845/1664] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:18:55.965841 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b6844-1b38-41aa-9dc1-be1ad660f0aa admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:55.996032 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-396eb9bf-847f-4626-86df-3721c3ab0d38 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:55.999187 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-396eb9bf-847f-4626-86df-3721c3ab0d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:56.004717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b6844-1b38-41aa-9dc1-be1ad660f0aa admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:56.026573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 820/1665] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:18:56.027723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7031475c-e5c5-4c3e-9982-f398d83bbdd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:56.029159 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031475c-e5c5-4c3e-9982-f398d83bbdd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:56.029868 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031475c-e5c5-4c3e-9982-f398d83bbdd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:56.030539 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031475c-e5c5-4c3e-9982-f398d83bbdd9 admin admin] PATH_INFO: `/v3/projects/e6488bf30f0d415e8729d000495af25f/users/0886ce47c12944c8be5ca15707abab0f/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:56.033772 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396eb9bf-847f-4626-86df-3721c3ab0d38 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:56.100719 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396eb9bf-847f-4626-86df-3721c3ab0d38 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:56.168440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7031475c-e5c5-4c3e-9982-f398d83bbdd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6488bf30f0d415e8729d000495af25f, user_id=0886ce47c12944c8be5ca15707abab0f, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:56.173332 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf704444-6885-4455-afbe-f46fe99ea531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:56.174937 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf704444-6885-4455-afbe-f46fe99ea531 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:56.175433 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf704444-6885-4455-afbe-f46fe99ea531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:56.175956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf704444-6885-4455-afbe-f46fe99ea531 admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:56.183075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7031475c-e5c5-4c3e-9982-f398d83bbdd9 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:56.195203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf704444-6885-4455-afbe-f46fe99ea531 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:18:56.195961 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf704444-6885-4455-afbe-f46fe99ea531 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:56.199088 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf704444-6885-4455-afbe-f46fe99ea531 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:56.283561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 846/1666] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:18:55 2022] PUT /identity/v3/projects/e6488bf30f0d415e8729d000495af25f/users/0886ce47c12944c8be5ca15707abab0f/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:18:56.286478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 821/1667] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:18:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:18:56.289875 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03919b67-b76f-4f2f-a811-546aa8f18761 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:56.290375 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03919b67-b76f-4f2f-a811-546aa8f18761 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:56.290869 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03919b67-b76f-4f2f-a811-546aa8f18761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:56.297030 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf704444-6885-4455-afbe-f46fe99ea531 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:56.349392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf704444-6885-4455-afbe-f46fe99ea531 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:56.438191 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-03919b67-b76f-4f2f-a811-546aa8f18761 None None] MFA Rules not processed for user `0886ce47c12944c8be5ca15707abab0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:56.440738 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-03919b67-b76f-4f2f-a811-546aa8f18761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:56.644064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059ddbc-9464-4530-97d2-e00a6a48a0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:56.645115 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059ddbc-9464-4530-97d2-e00a6a48a0c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:56.645596 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059ddbc-9464-4530-97d2-e00a6a48a0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:56.646076 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059ddbc-9464-4530-97d2-e00a6a48a0c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:56.647223 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059ddbc-9464-4530-97d2-e00a6a48a0c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:56.659126 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059ddbc-9464-4530-97d2-e00a6a48a0c9 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:56.767687 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 847/1668] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:56.779083 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03919b67-b76f-4f2f-a811-546aa8f18761 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:56.782221 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-03919b67-b76f-4f2f-a811-546aa8f18761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:56.788070 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:56.810130 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 822/1669] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:18:56 2022] POST /identity/v3/projects => generated 369 bytes in 514 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:18:56.811913 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059ddbc-9464-4530-97d2-e00a6a48a0c9 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:57.029644 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059ddbc-9464-4530-97d2-e00a6a48a0c9 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:57.034172 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:57.056169 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8295bd1-9537-489b-89dd-0403abaf3464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:57.057210 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8295bd1-9537-489b-89dd-0403abaf3464 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:57.057687 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8295bd1-9537-489b-89dd-0403abaf3464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:57.058160 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8295bd1-9537-489b-89dd-0403abaf3464 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:57.059216 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8295bd1-9537-489b-89dd-0403abaf3464 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:57.062329 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8295bd1-9537-489b-89dd-0403abaf3464 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:57.080844 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03919b67-b76f-4f2f-a811-546aa8f18761 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:57.151124 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 823/1670] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:18:56 2022] POST /identity/v3/users => generated 285 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:18:57.153313 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8295bd1-9537-489b-89dd-0403abaf3464 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:57.174320 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8295bd1-9537-489b-89dd-0403abaf3464 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:57.190154 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c6964-724e-4f72-8346-da50d1395370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:57.191640 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c6964-724e-4f72-8346-da50d1395370 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:57.193073 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c6964-724e-4f72-8346-da50d1395370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:57.193926 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c6964-724e-4f72-8346-da50d1395370 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:57.195270 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5c6964-724e-4f72-8346-da50d1395370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:57.198602 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5c6964-724e-4f72-8346-da50d1395370 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:57.254515 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c683a0-632a-4abd-9713-153b48a4e464 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:57.255279 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c683a0-632a-4abd-9713-153b48a4e464 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:57.255708 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c683a0-632a-4abd-9713-153b48a4e464 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:57.256126 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c683a0-632a-4abd-9713-153b48a4e464 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:57.270683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 824/1671] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:18:57.273772 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c6964-724e-4f72-8346-da50d1395370 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:57.332084 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c6964-724e-4f72-8346-da50d1395370 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:57.339438 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc73c2-0773-443d-b301-0790529fc5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:57.340848 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc73c2-0773-443d-b301-0790529fc5d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:57.341287 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc73c2-0773-443d-b301-0790529fc5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:57.341744 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc73c2-0773-443d-b301-0790529fc5d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:57.342587 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc73c2-0773-443d-b301-0790529fc5d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:57.344982 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc73c2-0773-443d-b301-0790529fc5d4 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:57.380916 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 825/1672] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:18:57.382864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc73c2-0773-443d-b301-0790529fc5d4 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:57.408495 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc73c2-0773-443d-b301-0790529fc5d4 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:57.424160 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e571e-bec1-4a71-83e8-5e24953d95d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:57.424861 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e571e-bec1-4a71-83e8-5e24953d95d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:57.425308 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e571e-bec1-4a71-83e8-5e24953d95d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:57.425771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e571e-bec1-4a71-83e8-5e24953d95d5 admin admin] PATH_INFO: `/v3/projects/2cdbc3729f934f2dbe4e741fa497f71b/users/cf51fc40612543ba958e03a15679d864/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:57.429778 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c683a0-632a-4abd-9713-153b48a4e464 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:57.435067 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c683a0-632a-4abd-9713-153b48a4e464 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:57.449568 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 848/1673] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:18:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 672 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:18:57.463120 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e571e-bec1-4a71-83e8-5e24953d95d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cdbc3729f934f2dbe4e741fa497f71b, user_id=cf51fc40612543ba958e03a15679d864, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:57.467960 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e571e-bec1-4a71-83e8-5e24953d95d5 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:57.525008 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 826/1674] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:18:57 2022] PUT /identity/v3/projects/2cdbc3729f934f2dbe4e741fa497f71b/users/cf51fc40612543ba958e03a15679d864/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:18:57.529854 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d207999-500e-4c76-b2e6-815cf482b63c None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:57.530889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d207999-500e-4c76-b2e6-815cf482b63c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:57.530889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d207999-500e-4c76-b2e6-815cf482b63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:57.585127 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-9d207999-500e-4c76-b2e6-815cf482b63c None None] MFA Rules not processed for user `cf51fc40612543ba958e03a15679d864`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:57.592890 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9d207999-500e-4c76-b2e6-815cf482b63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:57.770920 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e571e-bec1-4a71-83e8-5e24953d95d5 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:57.773339 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 849/1675] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:57.800517 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d207999-500e-4c76-b2e6-815cf482b63c None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:57.884516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:57.940777 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e571e-bec1-4a71-83e8-5e24953d95d5 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:57.959019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36da87-16b0-436e-a068-f053484ff0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:57.959696 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36da87-16b0-436e-a068-f053484ff0cb admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:57.959979 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36da87-16b0-436e-a068-f053484ff0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:57.960233 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36da87-16b0-436e-a068-f053484ff0cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:57.960903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36da87-16b0-436e-a068-f053484ff0cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:57.969409 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36da87-16b0-436e-a068-f053484ff0cb admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:57.980130 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d207999-500e-4c76-b2e6-815cf482b63c None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:57.995733 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09301c0-bae2-40f5-902f-c842c11dae84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:57.996434 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09301c0-bae2-40f5-902f-c842c11dae84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:57.996434 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09301c0-bae2-40f5-902f-c842c11dae84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:57.996693 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09301c0-bae2-40f5-902f-c842c11dae84 admin admin] PATH_INFO: `/v3/users/0886ce47c12944c8be5ca15707abab0f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:58.003388 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09301c0-bae2-40f5-902f-c842c11dae84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0886ce47c12944c8be5ca15707abab0f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:58.010595 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09301c0-bae2-40f5-902f-c842c11dae84 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:58.035111 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 827/1676] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:18:57 2022] POST /identity/v3/projects => generated 367 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:18:58.044132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36da87-16b0-436e-a068-f053484ff0cb admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:58.130831 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-f09301c0-bae2-40f5-902f-c842c11dae84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0886ce47c12944c8be5ca15707abab0f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:58.145197 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-f09301c0-bae2-40f5-902f-c842c11dae84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0886ce47c12944c8be5ca15707abab0f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:58.146034 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-f09301c0-bae2-40f5-902f-c842c11dae84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0886ce47c12944c8be5ca15707abab0f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:58.146652 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-f09301c0-bae2-40f5-902f-c842c11dae84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0886ce47c12944c8be5ca15707abab0f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:58.163919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:58.165673 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-f09301c0-bae2-40f5-902f-c842c11dae84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0886ce47c12944c8be5ca15707abab0f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:18:58.194170 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 850/1677] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:18:57 2022] DELETE /identity/v3/users/0886ce47c12944c8be5ca15707abab0f => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:18:58.202925 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2deb9a-8519-4307-96b1-e04eb5bde185 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:58.203216 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2deb9a-8519-4307-96b1-e04eb5bde185 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:58.203490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2deb9a-8519-4307-96b1-e04eb5bde185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:58.249597 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-6e2deb9a-8519-4307-96b1-e04eb5bde185 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:58.254654 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6e2deb9a-8519-4307-96b1-e04eb5bde185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:58.256524 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36da87-16b0-436e-a068-f053484ff0cb admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:58.394326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 851/1678] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:58.436977 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd6b94-468e-4d1f-b3f4-e882b398b0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:58.441176 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd6b94-468e-4d1f-b3f4-e882b398b0a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:58.441902 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd6b94-468e-4d1f-b3f4-e882b398b0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:58.442780 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd6b94-468e-4d1f-b3f4-e882b398b0a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:58.444519 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd6b94-468e-4d1f-b3f4-e882b398b0a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:58.448714 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd6b94-468e-4d1f-b3f4-e882b398b0a3 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:58.499585 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 828/1679] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:18:58 2022] POST /identity/v3/users => generated 290 bytes in 455 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:18:58.503143 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2deb9a-8519-4307-96b1-e04eb5bde185 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:58.505022 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6e2deb9a-8519-4307-96b1-e04eb5bde185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:58.518856 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2deb9a-8519-4307-96b1-e04eb5bde185 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:58.532595 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e8450-d6b1-4c2e-a649-90859661c579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:58.534500 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e8450-d6b1-4c2e-a649-90859661c579 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:58.534661 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e8450-d6b1-4c2e-a649-90859661c579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:58.538852 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e8450-d6b1-4c2e-a649-90859661c579 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:58.539716 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0e8450-d6b1-4c2e-a649-90859661c579 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:58.542326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0e8450-d6b1-4c2e-a649-90859661c579 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:58.564641 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 852/1680] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:18:58.567676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd6b94-468e-4d1f-b3f4-e882b398b0a3 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:58.589302 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd6b94-468e-4d1f-b3f4-e882b398b0a3 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:58.592258 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4400c8-0db1-4b18-9e36-fbd8a03d15cc None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:58.592883 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4400c8-0db1-4b18-9e36-fbd8a03d15cc None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:58.593922 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4400c8-0db1-4b18-9e36-fbd8a03d15cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:58.609869 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7728ef8-0d6c-445e-be30-c19deb3566ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:58.610634 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7728ef8-0d6c-445e-be30-c19deb3566ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:58.611458 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7728ef8-0d6c-445e-be30-c19deb3566ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:58.611773 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7728ef8-0d6c-445e-be30-c19deb3566ea admin admin] PATH_INFO: `/v3/projects/df79bbe3b8ef47dca701b0e11f7fcf3b/users/0b8fbd9d00af43d791e16735b22571e1/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:58.642150 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7728ef8-0d6c-445e-be30-c19deb3566ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=df79bbe3b8ef47dca701b0e11f7fcf3b, user_id=0b8fbd9d00af43d791e16735b22571e1, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:58.645006 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7728ef8-0d6c-445e-be30-c19deb3566ea admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:58.666203 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-ea4400c8-0db1-4b18-9e36-fbd8a03d15cc None None] MFA Rules not processed for user `6fb28ddd5511425a93c7b486d78a99eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:58.667847 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 829/1681] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:18:58 2022] PUT /identity/v3/projects/df79bbe3b8ef47dca701b0e11f7fcf3b/users/0b8fbd9d00af43d791e16735b22571e1/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:18:58.667979 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ea4400c8-0db1-4b18-9e36-fbd8a03d15cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:58.668883 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7728ef8-0d6c-445e-be30-c19deb3566ea admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:58.671147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-a7728ef8-0d6c-445e-be30-c19deb3566ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:58.726963 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:58.832507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 853/1682] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:58.834973 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4400c8-0db1-4b18-9e36-fbd8a03d15cc None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:58.836354 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7728ef8-0d6c-445e-be30-c19deb3566ea admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:58.845384 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22ebf6-9224-4bd5-8e95-b7baf2cf0a4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:58.846178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22ebf6-9224-4bd5-8e95-b7baf2cf0a4c service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:58.848596 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22ebf6-9224-4bd5-8e95-b7baf2cf0a4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:58.849041 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22ebf6-9224-4bd5-8e95-b7baf2cf0a4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:58.984266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:58.985161 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4400c8-0db1-4b18-9e36-fbd8a03d15cc None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:58.998902 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b934911-b244-4d19-954e-230e4dfee201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:58.999788 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b934911-b244-4d19-954e-230e4dfee201 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:59.000173 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b934911-b244-4d19-954e-230e4dfee201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:59.000549 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b934911-b244-4d19-954e-230e4dfee201 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:59.001398 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b934911-b244-4d19-954e-230e4dfee201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:59.002558 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf22ebf6-9224-4bd5-8e95-b7baf2cf0a4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:59.003987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b934911-b244-4d19-954e-230e4dfee201 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:59.004477 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf22ebf6-9224-4bd5-8e95-b7baf2cf0a4c service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:59.018625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 854/1683] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:18:59.019039 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 830/1684] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:18:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:18:59.020455 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215d8c8-5b0b-495f-9c42-b14d352437f9 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:59.020857 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215d8c8-5b0b-495f-9c42-b14d352437f9 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:59.021354 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215d8c8-5b0b-495f-9c42-b14d352437f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:59.022760 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22ebf6-9224-4bd5-8e95-b7baf2cf0a4c service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:59.037648 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22ebf6-9224-4bd5-8e95-b7baf2cf0a4c service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:59.044165 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-5215d8c8-5b0b-495f-9c42-b14d352437f9 None None] MFA Rules not processed for user `6fb28ddd5511425a93c7b486d78a99eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:59.045953 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5215d8c8-5b0b-495f-9c42-b14d352437f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:59.052628 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29efb560-6af7-4dc2-8ed3-c38f1288da18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:59.054480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efb560-6af7-4dc2-8ed3-c38f1288da18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:59.056211 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efb560-6af7-4dc2-8ed3-c38f1288da18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:59.056778 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efb560-6af7-4dc2-8ed3-c38f1288da18 admin admin] PATH_INFO: `/v3/projects/df79bbe3b8ef47dca701b0e11f7fcf3b/users/0b8fbd9d00af43d791e16735b22571e1/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:59.058039 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 855/1685] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:59.071410 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215d8c8-5b0b-495f-9c42-b14d352437f9 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:59.073504 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5215d8c8-5b0b-495f-9c42-b14d352437f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:59.074347 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29efb560-6af7-4dc2-8ed3-c38f1288da18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df79bbe3b8ef47dca701b0e11f7fcf3b, user_id=0b8fbd9d00af43d791e16735b22571e1, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:59.088572 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29efb560-6af7-4dc2-8ed3-c38f1288da18 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:59.130992 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 831/1686] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:18:59 2022] PUT /identity/v3/projects/df79bbe3b8ef47dca701b0e11f7fcf3b/users/0b8fbd9d00af43d791e16735b22571e1/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:18:59.145008 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a72326-7aed-4933-bc04-808bdbe07bd2 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:59.145294 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a72326-7aed-4933-bc04-808bdbe07bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:59.145571 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a72326-7aed-4933-bc04-808bdbe07bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:59.179654 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215d8c8-5b0b-495f-9c42-b14d352437f9 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:59.197586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-39a72326-7aed-4933-bc04-808bdbe07bd2 None None] MFA Rules not processed for user `0b8fbd9d00af43d791e16735b22571e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:18:59.199580 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-39a72326-7aed-4933-bc04-808bdbe07bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:59.273729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:18:59.303485 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c10790-30e4-46d8-b5c2-d78437d70b6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:59.304191 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c10790-30e4-46d8-b5c2-d78437d70b6a service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:59.304475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c10790-30e4-46d8-b5c2-d78437d70b6a service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:59.304751 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c10790-30e4-46d8-b5c2-d78437d70b6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:59.313949 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 832/1687] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:18:59 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:18:59.328640 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a72326-7aed-4933-bc04-808bdbe07bd2 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:18:59.334646 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-39a72326-7aed-4933-bc04-808bdbe07bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:18:59.425501 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c10790-30e4-46d8-b5c2-d78437d70b6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:59.427824 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c10790-30e4-46d8-b5c2-d78437d70b6a service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:59.459060 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 856/1688] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:18:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:18:59.463691 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a72326-7aed-4933-bc04-808bdbe07bd2 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:18:59.470699 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f991c93c-47c7-4cc9-8898-cbdf35736eaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:18:59.471452 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991c93c-47c7-4cc9-8898-cbdf35736eaf service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:18:59.471857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991c93c-47c7-4cc9-8898-cbdf35736eaf service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:18:59.472226 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991c93c-47c7-4cc9-8898-cbdf35736eaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:18:59.483824 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f991c93c-47c7-4cc9-8898-cbdf35736eaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:18:59.485489 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f991c93c-47c7-4cc9-8898-cbdf35736eaf service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:18:59.503745 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 833/1689] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:18:59 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:00.053827 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33ab39-a16b-4a67-8027-d89faf4d06a2 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:00.054353 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33ab39-a16b-4a67-8027-d89faf4d06a2 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:00.054868 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33ab39-a16b-4a67-8027-d89faf4d06a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:00.111835 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-4d33ab39-a16b-4a67-8027-d89faf4d06a2 None None] MFA Rules not processed for user `83d3b245c185424a9b6e5d69d691f031`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:00.114686 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4d33ab39-a16b-4a67-8027-d89faf4d06a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:00.273451 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2223245-db55-4e7c-a250-b145c5cd4de2 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:00.274270 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2223245-db55-4e7c-a250-b145c5cd4de2 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:00.274842 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2223245-db55-4e7c-a250-b145c5cd4de2 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:00.278219 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 834/1690] 10.222.0.16 () {54 vars in 912 bytes} [Mon Sep 5 12:19:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:19:00.311943 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deafae2a-60f9-4f7b-86ae-da8351e5cd70 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:00.312805 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deafae2a-60f9-4f7b-86ae-da8351e5cd70 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:00.313714 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deafae2a-60f9-4f7b-86ae-da8351e5cd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:00.431138 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-deafae2a-60f9-4f7b-86ae-da8351e5cd70 None None] MFA Rules not processed for user `9f79599ed0b74bdbab198627ef72252a`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:00.432531 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-deafae2a-60f9-4f7b-86ae-da8351e5cd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:00.451827 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 857/1691] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:00 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:00.565132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 835/1692] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Sep 5 12:19:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:19:00.600253 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33ab39-a16b-4a67-8027-d89faf4d06a2 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:00.600253 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4d33ab39-a16b-4a67-8027-d89faf4d06a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:00.648677 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deafae2a-60f9-4f7b-86ae-da8351e5cd70 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:00.660119 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-deafae2a-60f9-4f7b-86ae-da8351e5cd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:00.674474 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:00.680935 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33ab39-a16b-4a67-8027-d89faf4d06a2 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:00.704229 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3767634-a25b-4438-95ab-877475d4717e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:00.705314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3767634-a25b-4438-95ab-877475d4717e service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:00.705792 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3767634-a25b-4438-95ab-877475d4717e service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:00.706276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3767634-a25b-4438-95ab-877475d4717e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:00.782535 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3767634-a25b-4438-95ab-877475d4717e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:00.791956 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3767634-a25b-4438-95ab-877475d4717e service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:00.853466 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 858/1693] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:00.896537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deafae2a-60f9-4f7b-86ae-da8351e5cd70 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:00.926020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf143661-2547-402b-9d71-6eba98c718e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:00.927368 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf143661-2547-402b-9d71-6eba98c718e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:00.929612 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf143661-2547-402b-9d71-6eba98c718e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:00.931817 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf143661-2547-402b-9d71-6eba98c718e1 admin admin] PATH_INFO: `/v3/projects/e6488bf30f0d415e8729d000495af25f` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:00.938474 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf143661-2547-402b-9d71-6eba98c718e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6488bf30f0d415e8729d000495af25f)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:00.945703 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf143661-2547-402b-9d71-6eba98c718e1 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:01.084817 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-bf143661-2547-402b-9d71-6eba98c718e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6488bf30f0d415e8729d000495af25f', 'request_id': 'req-bf143661-2547-402b-9d71-6eba98c718e1'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:01.098788 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-bf143661-2547-402b-9d71-6eba98c718e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6488bf30f0d415e8729d000495af25f', 'request_id': 'req-bf143661-2547-402b-9d71-6eba98c718e1'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:01.115168 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-bf143661-2547-402b-9d71-6eba98c718e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6488bf30f0d415e8729d000495af25f', 'request_id': 'req-bf143661-2547-402b-9d71-6eba98c718e1'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:01.144205 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-bf143661-2547-402b-9d71-6eba98c718e1 admin admin] The token cache is being invalidate because project e6488bf30f0d415e8729d000495af25f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:01.145118 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-bf143661-2547-402b-9d71-6eba98c718e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:01.146696 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-bf143661-2547-402b-9d71-6eba98c718e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:01.150329 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 836/1694] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:00 2022] DELETE /identity/v3/projects/e6488bf30f0d415e8729d000495af25f => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:19:01.156738 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd1e59-bfcc-43d9-8890-5bd7c71a2516 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:01.157580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd1e59-bfcc-43d9-8890-5bd7c71a2516 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:01.157580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd1e59-bfcc-43d9-8890-5bd7c71a2516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:01.213114 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-7bcd1e59-bfcc-43d9-8890-5bd7c71a2516 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:01.214370 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7bcd1e59-bfcc-43d9-8890-5bd7c71a2516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:01.445670 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 859/1695] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:01.460110 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013f1a8-7a2f-4756-9e5b-236f1e1364fe None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:01.460699 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013f1a8-7a2f-4756-9e5b-236f1e1364fe None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:01.460699 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013f1a8-7a2f-4756-9e5b-236f1e1364fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:01.560557 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-5013f1a8-7a2f-4756-9e5b-236f1e1364fe None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:01.564727 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-5013f1a8-7a2f-4756-9e5b-236f1e1364fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:01.601124 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 837/1696] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:01.604449 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd1e59-bfcc-43d9-8890-5bd7c71a2516 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:01.616690 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd1e59-bfcc-43d9-8890-5bd7c71a2516 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:01.634565 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5dddc4-dece-4697-a2e9-6a3edbb9ae1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:01.637546 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5dddc4-dece-4697-a2e9-6a3edbb9ae1d admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:01.639700 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5dddc4-dece-4697-a2e9-6a3edbb9ae1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:01.640272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5dddc4-dece-4697-a2e9-6a3edbb9ae1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:01.641169 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5dddc4-dece-4697-a2e9-6a3edbb9ae1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:19:01.642436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5dddc4-dece-4697-a2e9-6a3edbb9ae1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:01.644377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5dddc4-dece-4697-a2e9-6a3edbb9ae1d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:01.700734 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 860/1697] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:19:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:19:01.708368 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013f1a8-7a2f-4756-9e5b-236f1e1364fe None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:01.733930 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013f1a8-7a2f-4756-9e5b-236f1e1364fe None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:01.758877 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b6177-726b-4663-9927-2e930bb19161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:01.759506 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b6177-726b-4663-9927-2e930bb19161 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:01.759873 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b6177-726b-4663-9927-2e930bb19161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:01.760148 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b6177-726b-4663-9927-2e930bb19161 admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:01.760812 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b6177-726b-4663-9927-2e930bb19161 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:01.763649 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b6177-726b-4663-9927-2e930bb19161 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:01.891362 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 838/1698] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:01 2022] POST /identity/v3/projects => generated 371 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:19:01.894879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5dddc4-dece-4697-a2e9-6a3edbb9ae1d admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:01.910980 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5dddc4-dece-4697-a2e9-6a3edbb9ae1d admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:01.926292 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:02.112733 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953cdb2-c5b9-46a0-a368-187853107b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:02.113950 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953cdb2-c5b9-46a0-a368-187853107b71 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:02.114240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953cdb2-c5b9-46a0-a368-187853107b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:02.114518 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953cdb2-c5b9-46a0-a368-187853107b71 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:02.115190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953cdb2-c5b9-46a0-a368-187853107b71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:02.117265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953cdb2-c5b9-46a0-a368-187853107b71 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:02.155729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 861/1699] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:01 2022] POST /identity/v3/users => generated 286 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:19:02.162893 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b6177-726b-4663-9927-2e930bb19161 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:02.187502 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b6177-726b-4663-9927-2e930bb19161 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:02.196290 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00a601-9d08-42d5-a23b-a090cf1e96f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:02.197621 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00a601-9d08-42d5-a23b-a090cf1e96f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:02.197907 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00a601-9d08-42d5-a23b-a090cf1e96f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:02.198185 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00a601-9d08-42d5-a23b-a090cf1e96f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:02.199096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a00a601-9d08-42d5-a23b-a090cf1e96f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:02.200787 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a00a601-9d08-42d5-a23b-a090cf1e96f9 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:02.239367 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 839/1700] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:02.254855 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953cdb2-c5b9-46a0-a368-187853107b71 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:02.274496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953cdb2-c5b9-46a0-a368-187853107b71 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:02.285635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3c6e3-053f-4669-bf7d-17fe61632ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:02.287218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3c6e3-053f-4669-bf7d-17fe61632ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:02.288142 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3c6e3-053f-4669-bf7d-17fe61632ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:02.289130 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3c6e3-053f-4669-bf7d-17fe61632ad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:02.291764 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3c6e3-053f-4669-bf7d-17fe61632ad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:02.297469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3c6e3-053f-4669-bf7d-17fe61632ad5 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:02.372733 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 862/1701] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:02.385671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00a601-9d08-42d5-a23b-a090cf1e96f9 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:02.417736 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00a601-9d08-42d5-a23b-a090cf1e96f9 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:02.433347 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c5766-4d0a-4e3e-9cf8-7141b74a1202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:02.445221 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c5766-4d0a-4e3e-9cf8-7141b74a1202 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:02.448189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c5766-4d0a-4e3e-9cf8-7141b74a1202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:02.449102 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c5766-4d0a-4e3e-9cf8-7141b74a1202 admin admin] PATH_INFO: `/v3/projects/c4f0c7a35c014e69bd36bc3dd7021844/users/b0ccc8224abe4acebfc579daaba9d4df/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:02.477733 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3c6e3-053f-4669-bf7d-17fe61632ad5 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:02.482818 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-58f3c6e3-053f-4669-bf7d-17fe61632ad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:02.516663 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c5766-4d0a-4e3e-9cf8-7141b74a1202 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4f0c7a35c014e69bd36bc3dd7021844, user_id=b0ccc8224abe4acebfc579daaba9d4df, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:02.525880 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c5766-4d0a-4e3e-9cf8-7141b74a1202 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:02.563399 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 840/1702] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:02 2022] PUT /identity/v3/projects/c4f0c7a35c014e69bd36bc3dd7021844/users/b0ccc8224abe4acebfc579daaba9d4df/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:02.577101 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f49ab1-fe59-46d5-8af9-eb0ce7f56f0f None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:02.582323 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f49ab1-fe59-46d5-8af9-eb0ce7f56f0f None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:02.592122 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f49ab1-fe59-46d5-8af9-eb0ce7f56f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:02.726671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-23f49ab1-fe59-46d5-8af9-eb0ce7f56f0f None None] MFA Rules not processed for user `b0ccc8224abe4acebfc579daaba9d4df`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:02.734770 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-23f49ab1-fe59-46d5-8af9-eb0ce7f56f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:02.819738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:02.852314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3c6e3-053f-4669-bf7d-17fe61632ad5 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:02.938125 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 841/1703] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:02.945496 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f49ab1-fe59-46d5-8af9-eb0ce7f56f0f None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:03.029444 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659a2a3-2d6b-48f9-91ec-ce606ae02833 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524bcfffcdfa4790885e0d4cbd66fa80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:03.030076 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659a2a3-2d6b-48f9-91ec-ce606ae02833 service placement] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:03.031296 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659a2a3-2d6b-48f9-91ec-ce606ae02833 service placement] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:03.032370 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659a2a3-2d6b-48f9-91ec-ce606ae02833 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:03.039822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b659a2a3-2d6b-48f9-91ec-ce606ae02833 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:03.069505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659a2a3-2d6b-48f9-91ec-ce606ae02833 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:03.073704 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659a2a3-2d6b-48f9-91ec-ce606ae02833 service placement] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:03.108157 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 863/1704] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:19:03.115277 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f49ab1-fe59-46d5-8af9-eb0ce7f56f0f None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:03.126021 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e73987-9e8c-49c8-91c8-5a9d6cc292ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:03.127228 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e73987-9e8c-49c8-91c8-5a9d6cc292ef admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:03.128234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e73987-9e8c-49c8-91c8-5a9d6cc292ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:03.130820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e73987-9e8c-49c8-91c8-5a9d6cc292ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:03.132117 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e73987-9e8c-49c8-91c8-5a9d6cc292ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:03.139072 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e73987-9e8c-49c8-91c8-5a9d6cc292ef admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:03.216341 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8f6b6-5fb5-467c-97a8-fd9311d86c7f None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:03.216631 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8f6b6-5fb5-467c-97a8-fd9311d86c7f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:03.216906 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8f6b6-5fb5-467c-97a8-fd9311d86c7f None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:03.218645 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 864/1705] 10.222.0.16 () {54 vars in 912 bytes} [Mon Sep 5 12:19:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:19:03.228840 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca02d7c-0b2d-4d16-8b83-f15e89efc85e None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:03.229132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca02d7c-0b2d-4d16-8b83-f15e89efc85e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:03.229406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca02d7c-0b2d-4d16-8b83-f15e89efc85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:03.254314 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 842/1706] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:02 2022] POST /identity/v3/projects => generated 369 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:03.258775 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e73987-9e8c-49c8-91c8-5a9d6cc292ef admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:03.280056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-7ca02d7c-0b2d-4d16-8b83-f15e89efc85e None None] MFA Rules not processed for user `9f79599ed0b74bdbab198627ef72252a`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:03.282335 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7ca02d7c-0b2d-4d16-8b83-f15e89efc85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:03.306965 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:03.502876 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 865/1707] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Sep 5 12:19:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:19:03.515989 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e73987-9e8c-49c8-91c8-5a9d6cc292ef admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:03.524462 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4756ba-8725-43a4-a9db-103ae9a37167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:03.525932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4756ba-8725-43a4-a9db-103ae9a37167 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:03.526628 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4756ba-8725-43a4-a9db-103ae9a37167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:03.527041 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4756ba-8725-43a4-a9db-103ae9a37167 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:03.528182 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4756ba-8725-43a4-a9db-103ae9a37167 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:03.530313 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4756ba-8725-43a4-a9db-103ae9a37167 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:03.601251 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 843/1708] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:03 2022] POST /identity/v3/users => generated 291 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:03.605832 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca02d7c-0b2d-4d16-8b83-f15e89efc85e None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:03.626058 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca02d7c-0b2d-4d16-8b83-f15e89efc85e None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:03.635432 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e89060-6b12-4b0b-ad6f-80f8bffe7e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:03.636768 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e89060-6b12-4b0b-ad6f-80f8bffe7e70 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:03.637330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e89060-6b12-4b0b-ad6f-80f8bffe7e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:03.637820 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e89060-6b12-4b0b-ad6f-80f8bffe7e70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:03.638732 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e89060-6b12-4b0b-ad6f-80f8bffe7e70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:03.642209 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e89060-6b12-4b0b-ad6f-80f8bffe7e70 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:03.680479 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 866/1709] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:03.682527 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4756ba-8725-43a4-a9db-103ae9a37167 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:03.697531 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4756ba-8725-43a4-a9db-103ae9a37167 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:03.704545 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6c630-4ba2-4fda-b1a1-6f8fafef4ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:03.705718 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6c630-4ba2-4fda-b1a1-6f8fafef4ff6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:03.706440 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6c630-4ba2-4fda-b1a1-6f8fafef4ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:03.707296 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6c630-4ba2-4fda-b1a1-6f8fafef4ff6 admin admin] PATH_INFO: `/v3/projects/1774836db0fa4ebebad29925812451dc/users/2d14bc0b2a7a487facbebdb843872e74/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:03.733738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6c630-4ba2-4fda-b1a1-6f8fafef4ff6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1774836db0fa4ebebad29925812451dc, user_id=2d14bc0b2a7a487facbebdb843872e74, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:03.738191 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6c630-4ba2-4fda-b1a1-6f8fafef4ff6 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:03.776941 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 844/1710] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:03 2022] PUT /identity/v3/projects/1774836db0fa4ebebad29925812451dc/users/2d14bc0b2a7a487facbebdb843872e74/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:03.777751 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e89060-6b12-4b0b-ad6f-80f8bffe7e70 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:03.789318 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6c630-4ba2-4fda-b1a1-6f8fafef4ff6 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:03.792627 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-b1c6c630-4ba2-4fda-b1a1-6f8fafef4ff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:03.801799 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e89060-6b12-4b0b-ad6f-80f8bffe7e70 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:03.842657 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:04.033906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6c630-4ba2-4fda-b1a1-6f8fafef4ff6 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:04.052493 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84435cc4-dad9-4bb5-a9c6-409dc383f335 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:04.053175 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84435cc4-dad9-4bb5-a9c6-409dc383f335 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:04.053476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84435cc4-dad9-4bb5-a9c6-409dc383f335 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:04.053750 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84435cc4-dad9-4bb5-a9c6-409dc383f335 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:04.068087 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84435cc4-dad9-4bb5-a9c6-409dc383f335 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:04.069578 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84435cc4-dad9-4bb5-a9c6-409dc383f335 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:04.102302 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 845/1711] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:04.103241 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84435cc4-dad9-4bb5-a9c6-409dc383f335 service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:04.111830 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-84435cc4-dad9-4bb5-a9c6-409dc383f335 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:04.120167 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b6d27-3f83-4219-8964-cd86ccb1dc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:04.121206 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b6d27-3f83-4219-8964-cd86ccb1dc72 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:04.121206 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b6d27-3f83-4219-8964-cd86ccb1dc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:04.121524 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b6d27-3f83-4219-8964-cd86ccb1dc72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:04.152506 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3b6d27-3f83-4219-8964-cd86ccb1dc72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:04.154638 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3b6d27-3f83-4219-8964-cd86ccb1dc72 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:04.165336 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84435cc4-dad9-4bb5-a9c6-409dc383f335 service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:04.176701 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 867/1712] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:04.183833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac0b667-7c50-43dc-a8a1-f99ab6a29d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:04.185422 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac0b667-7c50-43dc-a8a1-f99ab6a29d63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:04.185701 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac0b667-7c50-43dc-a8a1-f99ab6a29d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:04.185970 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac0b667-7c50-43dc-a8a1-f99ab6a29d63 admin admin] PATH_INFO: `/v3/users/cf51fc40612543ba958e03a15679d864` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:04.187152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b6d27-3f83-4219-8964-cd86ccb1dc72 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:04.187962 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac0b667-7c50-43dc-a8a1-f99ab6a29d63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf51fc40612543ba958e03a15679d864)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:04.197033 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac0b667-7c50-43dc-a8a1-f99ab6a29d63 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:04.268838 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b6d27-3f83-4219-8964-cd86ccb1dc72 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:04.285046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eecd6a-c314-4b88-bcd5-ed4aa02366e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:04.295262 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eecd6a-c314-4b88-bcd5-ed4aa02366e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:04.296153 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eecd6a-c314-4b88-bcd5-ed4aa02366e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:04.296647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eecd6a-c314-4b88-bcd5-ed4aa02366e9 admin admin] PATH_INFO: `/v3/projects/1774836db0fa4ebebad29925812451dc/users/2d14bc0b2a7a487facbebdb843872e74/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:04.299484 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eecd6a-c314-4b88-bcd5-ed4aa02366e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1774836db0fa4ebebad29925812451dc, user_id=2d14bc0b2a7a487facbebdb843872e74, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:04.305137 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eecd6a-c314-4b88-bcd5-ed4aa02366e9 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:04.322529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 868/1713] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:04 2022] PUT /identity/v3/projects/1774836db0fa4ebebad29925812451dc/users/2d14bc0b2a7a487facbebdb843872e74/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:04.341543 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b012a-ed68-4e7c-b398-a37fce134cd3 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:04.341929 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b012a-ed68-4e7c-b398-a37fce134cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:04.342308 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b012a-ed68-4e7c-b398-a37fce134cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:04.377010 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-7ac0b667-7c50-43dc-a8a1-f99ab6a29d63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf51fc40612543ba958e03a15679d864'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:04.408680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-7ac0b667-7c50-43dc-a8a1-f99ab6a29d63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf51fc40612543ba958e03a15679d864'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:04.409932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-7ac0b667-7c50-43dc-a8a1-f99ab6a29d63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf51fc40612543ba958e03a15679d864'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:04.448363 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-344b012a-ed68-4e7c-b398-a37fce134cd3 None None] MFA Rules not processed for user `2d14bc0b2a7a487facbebdb843872e74`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:04.448873 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-7ac0b667-7c50-43dc-a8a1-f99ab6a29d63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf51fc40612543ba958e03a15679d864'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:04.454429 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-344b012a-ed68-4e7c-b398-a37fce134cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:04.484843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-7ac0b667-7c50-43dc-a8a1-f99ab6a29d63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf51fc40612543ba958e03a15679d864'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:04.513416 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 846/1714] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:04 2022] DELETE /identity/v3/users/cf51fc40612543ba958e03a15679d864 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:04.520908 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69664d9d-e790-41ca-9bdd-5a4ed792b301 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:04.521920 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69664d9d-e790-41ca-9bdd-5a4ed792b301 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:04.524799 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69664d9d-e790-41ca-9bdd-5a4ed792b301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:04.588223 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-69664d9d-e790-41ca-9bdd-5a4ed792b301 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:04.589473 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-69664d9d-e790-41ca-9bdd-5a4ed792b301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:04.680688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 869/1715] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:04 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:04.732557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b012a-ed68-4e7c-b398-a37fce134cd3 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:04.733850 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-344b012a-ed68-4e7c-b398-a37fce134cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:04.764224 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 847/1716] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:04.775167 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69664d9d-e790-41ca-9bdd-5a4ed792b301 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:04.776953 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-69664d9d-e790-41ca-9bdd-5a4ed792b301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:04.794224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:04.834862 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:04.869694 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b012a-ed68-4e7c-b398-a37fce134cd3 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:04.905530 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69664d9d-e790-41ca-9bdd-5a4ed792b301 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:04.918754 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f386e2-4531-4788-9567-3d71c999b028 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:04.919943 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f386e2-4531-4788-9567-3d71c999b028 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:04.920231 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f386e2-4531-4788-9567-3d71c999b028 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:04.920542 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f386e2-4531-4788-9567-3d71c999b028 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:04.944452 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f386e2-4531-4788-9567-3d71c999b028 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:04.945927 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f386e2-4531-4788-9567-3d71c999b028 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:04.976581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 848/1717] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:19:04.999033 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e967f926-796e-4e1d-af46-f0f08ab0e225 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:05.000303 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967f926-796e-4e1d-af46-f0f08ab0e225 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:05.000593 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967f926-796e-4e1d-af46-f0f08ab0e225 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:05.000867 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967f926-796e-4e1d-af46-f0f08ab0e225 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:05.191091 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e967f926-796e-4e1d-af46-f0f08ab0e225 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:05.212646 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e967f926-796e-4e1d-af46-f0f08ab0e225 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:05.217463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04459148-c615-4fbc-b159-c4fd1d26037e None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:05.217738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04459148-c615-4fbc-b159-c4fd1d26037e None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:05.218063 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04459148-c615-4fbc-b159-c4fd1d26037e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:05.252246 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 870/1718] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:04 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:05.290623 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-04459148-c615-4fbc-b159-c4fd1d26037e None None] MFA Rules not processed for user `45bd61574c6c4ebb9d6fe0c87228065f`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:05.292413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-04459148-c615-4fbc-b159-c4fd1d26037e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:05.531240 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 849/1719] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:19:05.919261 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e967f926-796e-4e1d-af46-f0f08ab0e225 service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:05.921435 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e967f926-796e-4e1d-af46-f0f08ab0e225 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:06.051196 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04459148-c615-4fbc-b159-c4fd1d26037e None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:06.053292 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-04459148-c615-4fbc-b159-c4fd1d26037e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:06.095782 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04459148-c615-4fbc-b159-c4fd1d26037e None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:06.112305 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be889ba-08a1-4f00-980a-8d05f90018a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:06.113662 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be889ba-08a1-4f00-980a-8d05f90018a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:06.115007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be889ba-08a1-4f00-980a-8d05f90018a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:06.116432 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be889ba-08a1-4f00-980a-8d05f90018a0 admin admin] PATH_INFO: `/v3/projects/2cdbc3729f934f2dbe4e741fa497f71b` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:06.120020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be889ba-08a1-4f00-980a-8d05f90018a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cdbc3729f934f2dbe4e741fa497f71b)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:06.127149 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be889ba-08a1-4f00-980a-8d05f90018a0 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:06.250841 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e967f926-796e-4e1d-af46-f0f08ab0e225 service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:06.285447 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-5be889ba-08a1-4f00-980a-8d05f90018a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cdbc3729f934f2dbe4e741fa497f71b', 'request_id': 'req-5be889ba-08a1-4f00-980a-8d05f90018a0'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:06.318551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-5be889ba-08a1-4f00-980a-8d05f90018a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cdbc3729f934f2dbe4e741fa497f71b', 'request_id': 'req-5be889ba-08a1-4f00-980a-8d05f90018a0'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:06.333394 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:06.352179 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-5be889ba-08a1-4f00-980a-8d05f90018a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cdbc3729f934f2dbe4e741fa497f71b', 'request_id': 'req-5be889ba-08a1-4f00-980a-8d05f90018a0'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:06.383428 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-5be889ba-08a1-4f00-980a-8d05f90018a0 admin admin] The token cache is being invalidate because project 2cdbc3729f934f2dbe4e741fa497f71b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:06.383827 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-5be889ba-08a1-4f00-980a-8d05f90018a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:06.384682 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-5be889ba-08a1-4f00-980a-8d05f90018a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:06.388548 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 850/1720] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:06 2022] DELETE /identity/v3/projects/2cdbc3729f934f2dbe4e741fa497f71b => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:06.393709 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be889ba-08a1-4f00-980a-8d05f90018a0 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:06.396379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-5be889ba-08a1-4f00-980a-8d05f90018a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:06.419379 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8df5f4-be6a-43f2-8244-948a0d3c1bcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:06.421912 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8df5f4-be6a-43f2-8244-948a0d3c1bcd service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:06.422740 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8df5f4-be6a-43f2-8244-948a0d3c1bcd service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:06.424904 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8df5f4-be6a-43f2-8244-948a0d3c1bcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:06.436817 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9b8df5f4-be6a-43f2-8244-948a0d3c1bcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:06.530667 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:06.625442 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be889ba-08a1-4f00-980a-8d05f90018a0 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:06.636104 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8df5f4-be6a-43f2-8244-948a0d3c1bcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:06.636684 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ba419-661f-47e3-8aca-e4506f82af2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:06.641796 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ba419-661f-47e3-8aca-e4506f82af2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:06.641796 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ba419-661f-47e3-8aca-e4506f82af2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:06.641796 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ba419-661f-47e3-8aca-e4506f82af2a admin admin] PATH_INFO: `/v3/users/0b8fbd9d00af43d791e16735b22571e1` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:06.641796 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21ba419-661f-47e3-8aca-e4506f82af2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b8fbd9d00af43d791e16735b22571e1)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:06.645147 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8df5f4-be6a-43f2-8244-948a0d3c1bcd service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:06.654008 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21ba419-661f-47e3-8aca-e4506f82af2a admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:06.663033 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 871/1721] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 744 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:06.671839 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8df5f4-be6a-43f2-8244-948a0d3c1bcd service nova] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:06.672343 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9b8df5f4-be6a-43f2-8244-948a0d3c1bcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:06.702971 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8df5f4-be6a-43f2-8244-948a0d3c1bcd service nova] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:06.716108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ea084-3660-4ec1-8bc4-4d0b32a69118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:06.717015 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ea084-3660-4ec1-8bc4-4d0b32a69118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:06.717365 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ea084-3660-4ec1-8bc4-4d0b32a69118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:06.717647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ea084-3660-4ec1-8bc4-4d0b32a69118 admin admin] PATH_INFO: `/v3/users/b0ccc8224abe4acebfc579daaba9d4df` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:06.720982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ea084-3660-4ec1-8bc4-4d0b32a69118 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0ccc8224abe4acebfc579daaba9d4df)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:06.725215 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ea084-3660-4ec1-8bc4-4d0b32a69118 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:06.803484 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b21ba419-661f-47e3-8aca-e4506f82af2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b8fbd9d00af43d791e16735b22571e1'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:06.825751 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b21ba419-661f-47e3-8aca-e4506f82af2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b8fbd9d00af43d791e16735b22571e1'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:06.850464 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b21ba419-661f-47e3-8aca-e4506f82af2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b8fbd9d00af43d791e16735b22571e1'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:06.864163 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-777ea084-3660-4ec1-8bc4-4d0b32a69118 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0ccc8224abe4acebfc579daaba9d4df'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:06.940292 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-777ea084-3660-4ec1-8bc4-4d0b32a69118 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0ccc8224abe4acebfc579daaba9d4df'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:06.942147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b21ba419-661f-47e3-8aca-e4506f82af2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b8fbd9d00af43d791e16735b22571e1'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:06.943170 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-777ea084-3660-4ec1-8bc4-4d0b32a69118 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0ccc8224abe4acebfc579daaba9d4df'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:06.945336 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-777ea084-3660-4ec1-8bc4-4d0b32a69118 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0ccc8224abe4acebfc579daaba9d4df'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:07.005435 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b21ba419-661f-47e3-8aca-e4506f82af2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b8fbd9d00af43d791e16735b22571e1'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:07.039039 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-777ea084-3660-4ec1-8bc4-4d0b32a69118 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0ccc8224abe4acebfc579daaba9d4df'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:07.052009 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 851/1722] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:06 2022] DELETE /identity/v3/users/0b8fbd9d00af43d791e16735b22571e1 => generated 0 bytes in 658 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:07.056173 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ba419-661f-47e3-8aca-e4506f82af2a admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:07.065434 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-b21ba419-661f-47e3-8aca-e4506f82af2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:07.105578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 872/1723] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:06 2022] DELETE /identity/v3/users/b0ccc8224abe4acebfc579daaba9d4df => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:07.112729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ea084-3660-4ec1-8bc4-4d0b32a69118 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:07.121909 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-777ea084-3660-4ec1-8bc4-4d0b32a69118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:07.363309 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:07.379252 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ba419-661f-47e3-8aca-e4506f82af2a admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:07.466429 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ea084-3660-4ec1-8bc4-4d0b32a69118 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:07.497504 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2334e6-708f-4b73-be18-6b061231ae60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:07.499281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2334e6-708f-4b73-be18-6b061231ae60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:07.499628 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2334e6-708f-4b73-be18-6b061231ae60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:07.499913 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2334e6-708f-4b73-be18-6b061231ae60 admin admin] PATH_INFO: `/v3/users/34b23635a4a443eea96fb7cf0f7bc5d5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:07.513562 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2334e6-708f-4b73-be18-6b061231ae60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34b23635a4a443eea96fb7cf0f7bc5d5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:07.521598 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2334e6-708f-4b73-be18-6b061231ae60 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:07.612642 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f69e3-4cad-4ed9-a903-577715b4250a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:07.613319 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f69e3-4cad-4ed9-a903-577715b4250a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:07.613603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f69e3-4cad-4ed9-a903-577715b4250a admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:07.615803 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f69e3-4cad-4ed9-a903-577715b4250a admin admin] PATH_INFO: `/v3/users/02de21bffe154077866824e2033608fc` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:07.619447 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280f69e3-4cad-4ed9-a903-577715b4250a admin admin] RBAC: Authorizing `identity:delete_user(user_id=02de21bffe154077866824e2033608fc)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:07.626846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280f69e3-4cad-4ed9-a903-577715b4250a admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:07.647009 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:07.698017 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ee2334e6-708f-4b73-be18-6b061231ae60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34b23635a4a443eea96fb7cf0f7bc5d5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:07.730767 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ee2334e6-708f-4b73-be18-6b061231ae60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34b23635a4a443eea96fb7cf0f7bc5d5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:07.734517 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ee2334e6-708f-4b73-be18-6b061231ae60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34b23635a4a443eea96fb7cf0f7bc5d5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:07.737341 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ee2334e6-708f-4b73-be18-6b061231ae60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34b23635a4a443eea96fb7cf0f7bc5d5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:07.773244 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ee2334e6-708f-4b73-be18-6b061231ae60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34b23635a4a443eea96fb7cf0f7bc5d5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:07.820578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 873/1724] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:07 2022] DELETE /identity/v3/users/34b23635a4a443eea96fb7cf0f7bc5d5 => generated 0 bytes in 714 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:07.825666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892d2dc-d1e4-4cba-a9aa-18d4d4f09dd1 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:07.826335 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892d2dc-d1e4-4cba-a9aa-18d4d4f09dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:07.830092 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892d2dc-d1e4-4cba-a9aa-18d4d4f09dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:07.861312 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-280f69e3-4cad-4ed9-a903-577715b4250a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02de21bffe154077866824e2033608fc'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:07.880529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-6892d2dc-d1e4-4cba-a9aa-18d4d4f09dd1 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:07.898710 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6892d2dc-d1e4-4cba-a9aa-18d4d4f09dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:07.910229 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-280f69e3-4cad-4ed9-a903-577715b4250a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02de21bffe154077866824e2033608fc'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:07.913769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-280f69e3-4cad-4ed9-a903-577715b4250a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02de21bffe154077866824e2033608fc'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:07.974504 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-280f69e3-4cad-4ed9-a903-577715b4250a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02de21bffe154077866824e2033608fc'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:07.988748 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-280f69e3-4cad-4ed9-a903-577715b4250a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02de21bffe154077866824e2033608fc'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:08.003698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 852/1725] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:07 2022] DELETE /identity/v3/users/02de21bffe154077866824e2033608fc => generated 0 bytes in 951 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:08.006665 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e7f9f-6b87-43ed-824a-1b6367b4f824 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:08.008809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e7f9f-6b87-43ed-824a-1b6367b4f824 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:08.009370 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e7f9f-6b87-43ed-824a-1b6367b4f824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:08.028584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 874/1726] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:08.034167 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20dec06-982f-4ba1-a7b2-1bef9b2d619a None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:08.038411 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20dec06-982f-4ba1-a7b2-1bef9b2d619a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:08.038979 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20dec06-982f-4ba1-a7b2-1bef9b2d619a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:08.075145 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-7c3e7f9f-6b87-43ed-824a-1b6367b4f824 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:08.076519 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-7c3e7f9f-6b87-43ed-824a-1b6367b4f824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:08.255569 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-c20dec06-982f-4ba1-a7b2-1bef9b2d619a None None] MFA Rules not processed for user `8c5ae98336c441118d01f46a50f732d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:08.264680 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c20dec06-982f-4ba1-a7b2-1bef9b2d619a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:08.323254 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 853/1727] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:08.330006 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af111cd4-1157-48f0-ae67-735f932d08b9 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:08.330250 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af111cd4-1157-48f0-ae67-735f932d08b9 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:08.330715 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af111cd4-1157-48f0-ae67-735f932d08b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:08.414502 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:08.426911 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-af111cd4-1157-48f0-ae67-735f932d08b9 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:08.427985 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-af111cd4-1157-48f0-ae67-735f932d08b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:08.445217 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 854/1728] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:08.447135 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af111cd4-1157-48f0-ae67-735f932d08b9 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:08.452941 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-af111cd4-1157-48f0-ae67-735f932d08b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:08.552198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 875/1729] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Sep 5 12:19:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 519 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:08.553270 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20dec06-982f-4ba1-a7b2-1bef9b2d619a None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:08.554899 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c20dec06-982f-4ba1-a7b2-1bef9b2d619a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:08.666722 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20dec06-982f-4ba1-a7b2-1bef9b2d619a None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:08.669903 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d22a6-09d2-40bc-a86f-6d81b2cae18a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:08.671345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d22a6-09d2-40bc-a86f-6d81b2cae18a service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:08.671345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d22a6-09d2-40bc-a86f-6d81b2cae18a service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:08.671649 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d22a6-09d2-40bc-a86f-6d81b2cae18a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:08.692663 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d22a6-09d2-40bc-a86f-6d81b2cae18a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:08.696451 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d22a6-09d2-40bc-a86f-6d81b2cae18a service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:08.735691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 876/1730] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:08.737642 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d22a6-09d2-40bc-a86f-6d81b2cae18a service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:08.766140 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d22a6-09d2-40bc-a86f-6d81b2cae18a service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:08.776030 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af111cd4-1157-48f0-ae67-735f932d08b9 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:08.776636 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d86f04-4595-4c14-bda5-c4e7e1c691bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:08.777824 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d86f04-4595-4c14-bda5-c4e7e1c691bd service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:08.778288 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d86f04-4595-4c14-bda5-c4e7e1c691bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:08.779405 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d86f04-4595-4c14-bda5-c4e7e1c691bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:08.787196 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6153a648-1486-44f3-a414-90be7690cc8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:08.789905 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153a648-1486-44f3-a414-90be7690cc8e service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:08.791345 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153a648-1486-44f3-a414-90be7690cc8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:08.791954 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153a648-1486-44f3-a414-90be7690cc8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:08.807842 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d86f04-4595-4c14-bda5-c4e7e1c691bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:08.812486 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d86f04-4595-4c14-bda5-c4e7e1c691bd service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:08.824581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6153a648-1486-44f3-a414-90be7690cc8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:08.826092 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6153a648-1486-44f3-a414-90be7690cc8e service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:08.843906 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 877/1731] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:08.861881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 855/1732] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:08.901543 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d86f04-4595-4c14-bda5-c4e7e1c691bd service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:08.904436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-25d86f04-4595-4c14-bda5-c4e7e1c691bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:08.936319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d86f04-4595-4c14-bda5-c4e7e1c691bd service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:08.942058 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6153a648-1486-44f3-a414-90be7690cc8e service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:08.944510 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-6153a648-1486-44f3-a414-90be7690cc8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:08.979871 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2dbfe7-86a0-43a8-80c2-4f7c55e18e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:08.980563 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2dbfe7-86a0-43a8-80c2-4f7c55e18e99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:08.981126 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2dbfe7-86a0-43a8-80c2-4f7c55e18e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:08.981432 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2dbfe7-86a0-43a8-80c2-4f7c55e18e99 admin admin] PATH_INFO: `/v3/projects/df79bbe3b8ef47dca701b0e11f7fcf3b` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:08.982885 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2dbfe7-86a0-43a8-80c2-4f7c55e18e99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df79bbe3b8ef47dca701b0e11f7fcf3b)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:08.995134 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: WARNING py.warnings [None req-ac2dbfe7-86a0-43a8-80c2-4f7c55e18e99 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 12:19:08.995134 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: warnings.warn(msg) Sep 05 12:19:08.995134 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: Sep 05 12:19:08.996627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2dbfe7-86a0-43a8-80c2-4f7c55e18e99 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:09.053097 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:09.150586 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ac2dbfe7-86a0-43a8-80c2-4f7c55e18e99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df79bbe3b8ef47dca701b0e11f7fcf3b', 'request_id': 'req-ac2dbfe7-86a0-43a8-80c2-4f7c55e18e99'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:09.170276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ac2dbfe7-86a0-43a8-80c2-4f7c55e18e99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df79bbe3b8ef47dca701b0e11f7fcf3b', 'request_id': 'req-ac2dbfe7-86a0-43a8-80c2-4f7c55e18e99'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:09.199203 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ac2dbfe7-86a0-43a8-80c2-4f7c55e18e99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df79bbe3b8ef47dca701b0e11f7fcf3b', 'request_id': 'req-ac2dbfe7-86a0-43a8-80c2-4f7c55e18e99'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:09.249901 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ac2dbfe7-86a0-43a8-80c2-4f7c55e18e99 admin admin] The token cache is being invalidate because project df79bbe3b8ef47dca701b0e11f7fcf3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:09.249901 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ac2dbfe7-86a0-43a8-80c2-4f7c55e18e99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:09.249901 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ac2dbfe7-86a0-43a8-80c2-4f7c55e18e99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:09.252489 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 878/1733] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:08 2022] DELETE /identity/v3/projects/df79bbe3b8ef47dca701b0e11f7fcf3b => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:09.259822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814c39f-00b0-4449-a807-7ba1f258dde9 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:09.261324 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814c39f-00b0-4449-a807-7ba1f258dde9 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:09.261625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814c39f-00b0-4449-a807-7ba1f258dde9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:09.316812 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-a814c39f-00b0-4449-a807-7ba1f258dde9 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:09.318094 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a814c39f-00b0-4449-a807-7ba1f258dde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:09.372236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:09.420790 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 879/1734] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:09.421764 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6153a648-1486-44f3-a414-90be7690cc8e service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:09.422801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0490512-61c4-4766-a3af-b5e6e0afeb4a None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:09.423898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0490512-61c4-4766-a3af-b5e6e0afeb4a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:09.424544 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0490512-61c4-4766-a3af-b5e6e0afeb4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:09.455744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-a0490512-61c4-4766-a3af-b5e6e0afeb4a None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:09.456682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a0490512-61c4-4766-a3af-b5e6e0afeb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:09.485575 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 880/1735] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:09.488137 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0490512-61c4-4766-a3af-b5e6e0afeb4a None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:09.510346 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0490512-61c4-4766-a3af-b5e6e0afeb4a None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:09.524974 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba4e35-af00-4fde-82d8-4051f2f0c382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:09.525617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba4e35-af00-4fde-82d8-4051f2f0c382 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:09.526152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba4e35-af00-4fde-82d8-4051f2f0c382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:09.526152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba4e35-af00-4fde-82d8-4051f2f0c382 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:09.526830 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba4e35-af00-4fde-82d8-4051f2f0c382 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:19:09.527163 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba4e35-af00-4fde-82d8-4051f2f0c382 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:09.535288 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba4e35-af00-4fde-82d8-4051f2f0c382 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:09.584260 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 881/1736] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:19:09.585801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba4e35-af00-4fde-82d8-4051f2f0c382 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:09.606058 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba4e35-af00-4fde-82d8-4051f2f0c382 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:09.612147 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd041f23-893a-449a-b0d2-374f36fa483a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:09.612908 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd041f23-893a-449a-b0d2-374f36fa483a admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:09.613183 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd041f23-893a-449a-b0d2-374f36fa483a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:09.613483 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd041f23-893a-449a-b0d2-374f36fa483a admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:09.614867 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd041f23-893a-449a-b0d2-374f36fa483a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:09.617088 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd041f23-893a-449a-b0d2-374f36fa483a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:09.657735 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33daa6-e283-4548-b576-15f6f165c815 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '857eb7657e7d45938baca17f91a16901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:09.658775 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33daa6-e283-4548-b576-15f6f165c815 service swift] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:09.659265 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33daa6-e283-4548-b576-15f6f165c815 service swift] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:09.659969 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33daa6-e283-4548-b576-15f6f165c815 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:09.660608 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 882/1737] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:09 2022] POST /identity/v3/projects => generated 379 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:09.666758 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd041f23-893a-449a-b0d2-374f36fa483a admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:09.675175 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-cb33daa6-e283-4548-b576-15f6f165c815 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:09.847256 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd041f23-893a-449a-b0d2-374f36fa483a admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:09.873191 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e54434-2225-4768-a03c-0d02e3ec3cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:09.873840 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e54434-2225-4768-a03c-0d02e3ec3cea admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:09.874122 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e54434-2225-4768-a03c-0d02e3ec3cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:09.874396 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e54434-2225-4768-a03c-0d02e3ec3cea admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:09.875086 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e54434-2225-4768-a03c-0d02e3ec3cea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:09.877203 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e54434-2225-4768-a03c-0d02e3ec3cea admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:09.923316 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33daa6-e283-4548-b576-15f6f165c815 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:09.926997 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33daa6-e283-4548-b576-15f6f165c815 service swift] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:09.942406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 883/1738] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:09 2022] POST /identity/v3/users => generated 290 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:09.950273 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e54434-2225-4768-a03c-0d02e3ec3cea admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:09.961503 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 856/1739] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Sep 5 12:19:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 1020 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 05 12:19:09.987965 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e54434-2225-4768-a03c-0d02e3ec3cea admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:09.999378 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c8e74-0f1c-45ce-8c90-57e1d39e52ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:10.000398 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c8e74-0f1c-45ce-8c90-57e1d39e52ba admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:10.000398 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c8e74-0f1c-45ce-8c90-57e1d39e52ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:10.000596 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c8e74-0f1c-45ce-8c90-57e1d39e52ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:10.001213 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c8e74-0f1c-45ce-8c90-57e1d39e52ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:10.007856 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c8e74-0f1c-45ce-8c90-57e1d39e52ba admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:10.095676 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 884/1740] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:10.101477 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33daa6-e283-4548-b576-15f6f165c815 service swift] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:10.132282 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33daa6-e283-4548-b576-15f6f165c815 service swift] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:10.163577 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73fa99-b1ac-4eb0-a938-1691b3e7633b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:10.165812 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73fa99-b1ac-4eb0-a938-1691b3e7633b admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:10.166588 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73fa99-b1ac-4eb0-a938-1691b3e7633b admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:10.169815 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73fa99-b1ac-4eb0-a938-1691b3e7633b admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:10.172921 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c73fa99-b1ac-4eb0-a938-1691b3e7633b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:10.177841 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c73fa99-b1ac-4eb0-a938-1691b3e7633b admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:10.219968 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 857/1741] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:10.222844 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c8e74-0f1c-45ce-8c90-57e1d39e52ba admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:10.256857 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c8e74-0f1c-45ce-8c90-57e1d39e52ba admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:10.284152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5bb36-0695-47e9-83cc-2f06af2f463d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:10.286533 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5bb36-0695-47e9-83cc-2f06af2f463d admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:10.287361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5bb36-0695-47e9-83cc-2f06af2f463d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:10.288123 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5bb36-0695-47e9-83cc-2f06af2f463d admin admin] PATH_INFO: `/v3/projects/b30af7a2c7264080a2dc2d80c17b07a9/users/8a50f9ac6fda4544980bcd1009ee80a9/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:10.342758 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc5bb36-0695-47e9-83cc-2f06af2f463d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b30af7a2c7264080a2dc2d80c17b07a9, user_id=8a50f9ac6fda4544980bcd1009ee80a9, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:10.351939 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc5bb36-0695-47e9-83cc-2f06af2f463d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:10.437842 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 885/1742] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:10 2022] PUT /identity/v3/projects/b30af7a2c7264080a2dc2d80c17b07a9/users/8a50f9ac6fda4544980bcd1009ee80a9/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:10.461013 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbeaa0a-e751-41da-8797-181026af0165 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:10.461329 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbeaa0a-e751-41da-8797-181026af0165 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:10.461576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbeaa0a-e751-41da-8797-181026af0165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:10.548090 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-5fbeaa0a-e751-41da-8797-181026af0165 None None] MFA Rules not processed for user `8a50f9ac6fda4544980bcd1009ee80a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:10.550159 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-5fbeaa0a-e751-41da-8797-181026af0165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:10.652581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:10.784644 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 858/1743] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:10.786844 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5bb36-0695-47e9-83cc-2f06af2f463d admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:10.979515 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:10.991512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5bb36-0695-47e9-83cc-2f06af2f463d admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:11.020063 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434ee8d-ec66-4888-9ea6-df3f13932140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:11.033287 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434ee8d-ec66-4888-9ea6-df3f13932140 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:11.036645 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434ee8d-ec66-4888-9ea6-df3f13932140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:11.037134 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434ee8d-ec66-4888-9ea6-df3f13932140 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:11.038382 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434ee8d-ec66-4888-9ea6-df3f13932140 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:11.044043 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434ee8d-ec66-4888-9ea6-df3f13932140 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:11.184897 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbeaa0a-e751-41da-8797-181026af0165 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:11.185742 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 886/1744] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:10 2022] POST /identity/v3/projects => generated 377 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:11.211281 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbeaa0a-e751-41da-8797-181026af0165 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:11.285182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987288a-98e3-4f81-8c66-88c2ef79e8e4 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:11.286314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987288a-98e3-4f81-8c66-88c2ef79e8e4 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:11.286822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987288a-98e3-4f81-8c66-88c2ef79e8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:11.361371 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-d987288a-98e3-4f81-8c66-88c2ef79e8e4 None None] MFA Rules not processed for user `9f79599ed0b74bdbab198627ef72252a`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:11.371250 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d987288a-98e3-4f81-8c66-88c2ef79e8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:11.721683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2994bf1-7de7-4377-8278-7092574ff556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:11.728455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2994bf1-7de7-4377-8278-7092574ff556 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:11.728830 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2994bf1-7de7-4377-8278-7092574ff556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:11.729137 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2994bf1-7de7-4377-8278-7092574ff556 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:11.729889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2994bf1-7de7-4377-8278-7092574ff556 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:11.743499 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2994bf1-7de7-4377-8278-7092574ff556 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:11.823012 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 887/1745] 10.222.0.55 () {58 vars in 1015 bytes} [Mon Sep 5 12:19:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 539 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:19:11.832863 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 859/1746] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:11 2022] POST /identity/v3/users => generated 289 bytes in 649 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:11.837136 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987288a-98e3-4f81-8c66-88c2ef79e8e4 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:11.857498 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987288a-98e3-4f81-8c66-88c2ef79e8e4 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:11.870899 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2994bf1-7de7-4377-8278-7092574ff556 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:11.872918 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-d2994bf1-7de7-4377-8278-7092574ff556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:11.887134 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7262979-b2bf-463d-a814-a4bd1548cc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:11.888191 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7262979-b2bf-463d-a814-a4bd1548cc2e admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:11.888468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7262979-b2bf-463d-a814-a4bd1548cc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:11.888735 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7262979-b2bf-463d-a814-a4bd1548cc2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:11.889431 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7262979-b2bf-463d-a814-a4bd1548cc2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:11.891145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7262979-b2bf-463d-a814-a4bd1548cc2e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:11.900956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2994bf1-7de7-4377-8278-7092574ff556 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:11.924752 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 888/1747] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:11.928463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673184e-9051-4316-9076-b3eb95d2918f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:11.929107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673184e-9051-4316-9076-b3eb95d2918f service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:11.929401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673184e-9051-4316-9076-b3eb95d2918f service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:11.929656 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673184e-9051-4316-9076-b3eb95d2918f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:11.942958 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7262979-b2bf-463d-a814-a4bd1548cc2e admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:11.949109 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2673184e-9051-4316-9076-b3eb95d2918f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:11.950708 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2673184e-9051-4316-9076-b3eb95d2918f service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:11.980885 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7262979-b2bf-463d-a814-a4bd1548cc2e admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:11.995220 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 860/1748] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:12.003649 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44810a-dc1a-4940-bf26-f71d48610eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:12.004505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44810a-dc1a-4940-bf26-f71d48610eba admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:12.004816 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44810a-dc1a-4940-bf26-f71d48610eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:12.005091 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44810a-dc1a-4940-bf26-f71d48610eba admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:12.005767 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b44810a-dc1a-4940-bf26-f71d48610eba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:12.012802 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b44810a-dc1a-4940-bf26-f71d48610eba admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:12.046650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673184e-9051-4316-9076-b3eb95d2918f service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:12.056807 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 889/1749] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:12.073145 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673184e-9051-4316-9076-b3eb95d2918f service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:12.093703 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8db07-630d-43b5-be05-6e471a538a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:12.094508 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8db07-630d-43b5-be05-6e471a538a22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:12.094792 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8db07-630d-43b5-be05-6e471a538a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:12.095755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8db07-630d-43b5-be05-6e471a538a22 admin admin] PATH_INFO: `/v3/projects/0fab995ba2e04d19bf15930702a40d96/users/12cd5402bd0442b0b2e71e7f933f3518/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:12.136486 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44810a-dc1a-4940-bf26-f71d48610eba admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:12.139021 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2b44810a-dc1a-4940-bf26-f71d48610eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:12.192356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44810a-dc1a-4940-bf26-f71d48610eba admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:12.218079 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f8db07-630d-43b5-be05-6e471a538a22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fab995ba2e04d19bf15930702a40d96, user_id=12cd5402bd0442b0b2e71e7f933f3518, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:12.227251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d282ca4-3a0d-4529-bb04-523d5cfdab16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:12.228031 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d282ca4-3a0d-4529-bb04-523d5cfdab16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:12.228297 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d282ca4-3a0d-4529-bb04-523d5cfdab16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:12.228576 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d282ca4-3a0d-4529-bb04-523d5cfdab16 admin admin] PATH_INFO: `/v3/projects/09d82afaf78a497395c3b4ba0b8294fd` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:12.234091 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d282ca4-3a0d-4529-bb04-523d5cfdab16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09d82afaf78a497395c3b4ba0b8294fd)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:12.235559 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f8db07-630d-43b5-be05-6e471a538a22 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:12.237151 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d282ca4-3a0d-4529-bb04-523d5cfdab16 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:12.309842 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 861/1750] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:12 2022] PUT /identity/v3/projects/0fab995ba2e04d19bf15930702a40d96/users/12cd5402bd0442b0b2e71e7f933f3518/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:12.331229 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db6d5b-5aad-4dc7-b5ab-3723200cb23c None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:12.332168 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db6d5b-5aad-4dc7-b5ab-3723200cb23c None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:12.332646 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db6d5b-5aad-4dc7-b5ab-3723200cb23c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:12.413850 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-22db6d5b-5aad-4dc7-b5ab-3723200cb23c None None] MFA Rules not processed for user `12cd5402bd0442b0b2e71e7f933f3518`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:12.415901 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-22db6d5b-5aad-4dc7-b5ab-3723200cb23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:12.426618 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4d282ca4-3a0d-4529-bb04-523d5cfdab16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09d82afaf78a497395c3b4ba0b8294fd', 'request_id': 'req-4d282ca4-3a0d-4529-bb04-523d5cfdab16'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:12.476900 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4d282ca4-3a0d-4529-bb04-523d5cfdab16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09d82afaf78a497395c3b4ba0b8294fd', 'request_id': 'req-4d282ca4-3a0d-4529-bb04-523d5cfdab16'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:12.507475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4d282ca4-3a0d-4529-bb04-523d5cfdab16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09d82afaf78a497395c3b4ba0b8294fd', 'request_id': 'req-4d282ca4-3a0d-4529-bb04-523d5cfdab16'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:12.543764 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4d282ca4-3a0d-4529-bb04-523d5cfdab16 admin admin] The token cache is being invalidate because project 09d82afaf78a497395c3b4ba0b8294fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:12.546020 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4d282ca4-3a0d-4529-bb04-523d5cfdab16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:12.552145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4d282ca4-3a0d-4529-bb04-523d5cfdab16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:12.558334 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 890/1751] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:12 2022] DELETE /identity/v3/projects/09d82afaf78a497395c3b4ba0b8294fd => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:12.566110 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6bf63-e3db-4043-9582-4ef519db0af4 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:12.566565 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6bf63-e3db-4043-9582-4ef519db0af4 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:12.567038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6bf63-e3db-4043-9582-4ef519db0af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:12.630612 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-95a6bf63-e3db-4043-9582-4ef519db0af4 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:12.643094 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-95a6bf63-e3db-4043-9582-4ef519db0af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:12.663991 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 862/1752] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:19:12.668439 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db6d5b-5aad-4dc7-b5ab-3723200cb23c None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:12.670987 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-22db6d5b-5aad-4dc7-b5ab-3723200cb23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:12.701778 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:12.769875 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:12.849886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 891/1753] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:12.851816 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a6bf63-e3db-4043-9582-4ef519db0af4 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:12.858243 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-95a6bf63-e3db-4043-9582-4ef519db0af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:12.897354 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db6d5b-5aad-4dc7-b5ab-3723200cb23c None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:12.916174 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3601b16-3f96-4ed3-a555-45170ec08f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:12.918325 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3601b16-3f96-4ed3-a555-45170ec08f28 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:12.919362 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3601b16-3f96-4ed3-a555-45170ec08f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:12.919488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a6bf63-e3db-4043-9582-4ef519db0af4 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:12.919843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3601b16-3f96-4ed3-a555-45170ec08f28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:12.921637 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3601b16-3f96-4ed3-a555-45170ec08f28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:12.925629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3601b16-3f96-4ed3-a555-45170ec08f28 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:12.933647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a72c7-2e24-4f7a-9935-a622af951ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:12.935579 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a72c7-2e24-4f7a-9935-a622af951ee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:12.936464 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a72c7-2e24-4f7a-9935-a622af951ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:12.938085 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a72c7-2e24-4f7a-9935-a622af951ee8 admin admin] PATH_INFO: `/v3/users/b5dfea46fd2f4f9e8111117b6852d3c7` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:12.942700 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a72c7-2e24-4f7a-9935-a622af951ee8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5dfea46fd2f4f9e8111117b6852d3c7)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:12.956591 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a72c7-2e24-4f7a-9935-a622af951ee8 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:13.031664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 863/1754] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:12 2022] POST /identity/v3/projects => generated 379 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:13.037312 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd240f-0004-47bd-b09c-271dfc182bad None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:13.038119 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd240f-0004-47bd-b09c-271dfc182bad None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:13.039295 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd240f-0004-47bd-b09c-271dfc182bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:13.103283 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-eafd240f-0004-47bd-b09c-271dfc182bad None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:13.106175 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-eafd240f-0004-47bd-b09c-271dfc182bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:13.108269 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6b0a72c7-2e24-4f7a-9935-a622af951ee8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5dfea46fd2f4f9e8111117b6852d3c7'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:13.145457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6b0a72c7-2e24-4f7a-9935-a622af951ee8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5dfea46fd2f4f9e8111117b6852d3c7'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:13.148360 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6b0a72c7-2e24-4f7a-9935-a622af951ee8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5dfea46fd2f4f9e8111117b6852d3c7'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:13.151590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6b0a72c7-2e24-4f7a-9935-a622af951ee8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5dfea46fd2f4f9e8111117b6852d3c7'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:13.193270 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6b0a72c7-2e24-4f7a-9935-a622af951ee8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5dfea46fd2f4f9e8111117b6852d3c7'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:13.226813 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 892/1755] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:12 2022] DELETE /identity/v3/users/b5dfea46fd2f4f9e8111117b6852d3c7 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:13.229644 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a72c7-2e24-4f7a-9935-a622af951ee8 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:13.235503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6b0a72c7-2e24-4f7a-9935-a622af951ee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:13.308888 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 864/1756] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:13.313832 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337e439-1b56-42c4-a3bf-607d44bfdaae None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:13.314401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337e439-1b56-42c4-a3bf-607d44bfdaae None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:13.315747 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337e439-1b56-42c4-a3bf-607d44bfdaae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:13.376282 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-b337e439-1b56-42c4-a3bf-607d44bfdaae None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:13.382107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-b337e439-1b56-42c4-a3bf-607d44bfdaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:13.430214 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a72c7-2e24-4f7a-9935-a622af951ee8 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:13.448092 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff72ff05-3488-43ee-b676-60ba80513655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:13.448757 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72ff05-3488-43ee-b676-60ba80513655 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:13.449203 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72ff05-3488-43ee-b676-60ba80513655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:13.449590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72ff05-3488-43ee-b676-60ba80513655 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:13.450514 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff72ff05-3488-43ee-b676-60ba80513655 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:13.457119 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff72ff05-3488-43ee-b676-60ba80513655 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:13.526638 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 893/1757] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:13 2022] POST /identity/v3/users => generated 296 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:13.528267 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff72ff05-3488-43ee-b676-60ba80513655 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:13.554343 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff72ff05-3488-43ee-b676-60ba80513655 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:13.568058 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1470bb-69d1-4b3c-baad-880de9823af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:13.569561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1470bb-69d1-4b3c-baad-880de9823af9 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:13.575100 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1470bb-69d1-4b3c-baad-880de9823af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:13.575587 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1470bb-69d1-4b3c-baad-880de9823af9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:13.576612 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 865/1758] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:13.577951 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1470bb-69d1-4b3c-baad-880de9823af9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:19:13.578601 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337e439-1b56-42c4-a3bf-607d44bfdaae None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:13.579208 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1470bb-69d1-4b3c-baad-880de9823af9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:13.581974 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1470bb-69d1-4b3c-baad-880de9823af9 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:13.583039 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-b337e439-1b56-42c4-a3bf-607d44bfdaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:13.640109 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337e439-1b56-42c4-a3bf-607d44bfdaae None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:13.646075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 894/1759] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:19:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:19:13.647286 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1470bb-69d1-4b3c-baad-880de9823af9 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:13.653324 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebb67e-fcc3-40d5-92f5-026364f734c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:13.658115 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebb67e-fcc3-40d5-92f5-026364f734c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:13.658614 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebb67e-fcc3-40d5-92f5-026364f734c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:13.659143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebb67e-fcc3-40d5-92f5-026364f734c4 admin admin] PATH_INFO: `/v3/projects/82831792f11f4dcd9770f009448e796e` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:13.661035 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ebb67e-fcc3-40d5-92f5-026364f734c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82831792f11f4dcd9770f009448e796e)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:13.664563 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ebb67e-fcc3-40d5-92f5-026364f734c4 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:13.681016 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1470bb-69d1-4b3c-baad-880de9823af9 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:13.689965 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40c9ac-aa80-4117-9ab1-81cd6f3fdc63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:13.691089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40c9ac-aa80-4117-9ab1-81cd6f3fdc63 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:13.691340 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40c9ac-aa80-4117-9ab1-81cd6f3fdc63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:13.691786 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40c9ac-aa80-4117-9ab1-81cd6f3fdc63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:13.692532 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a40c9ac-aa80-4117-9ab1-81cd6f3fdc63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:13.694297 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a40c9ac-aa80-4117-9ab1-81cd6f3fdc63 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:13.730066 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 895/1760] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:13.732507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40c9ac-aa80-4117-9ab1-81cd6f3fdc63 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:13.750307 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40c9ac-aa80-4117-9ab1-81cd6f3fdc63 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:13.755425 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:13.769830 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1fd96-badb-4ce7-ba3c-e5a731e3e84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:13.770673 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1fd96-badb-4ce7-ba3c-e5a731e3e84c admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:13.772155 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1fd96-badb-4ce7-ba3c-e5a731e3e84c admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:13.773202 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1fd96-badb-4ce7-ba3c-e5a731e3e84c admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:13.776268 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d1fd96-badb-4ce7-ba3c-e5a731e3e84c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:13.783206 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d1fd96-badb-4ce7-ba3c-e5a731e3e84c admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:13.797796 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-22ebb67e-fcc3-40d5-92f5-026364f734c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82831792f11f4dcd9770f009448e796e', 'request_id': 'req-22ebb67e-fcc3-40d5-92f5-026364f734c4'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:13.818330 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-22ebb67e-fcc3-40d5-92f5-026364f734c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82831792f11f4dcd9770f009448e796e', 'request_id': 'req-22ebb67e-fcc3-40d5-92f5-026364f734c4'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:13.838739 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-22ebb67e-fcc3-40d5-92f5-026364f734c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82831792f11f4dcd9770f009448e796e', 'request_id': 'req-22ebb67e-fcc3-40d5-92f5-026364f734c4'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:13.878673 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 896/1761] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:13 2022] POST /identity/v3/projects => generated 391 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:13.882683 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1fd96-badb-4ce7-ba3c-e5a731e3e84c admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:13.886502 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e7d1fd96-badb-4ce7-ba3c-e5a731e3e84c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:13.891505 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-22ebb67e-fcc3-40d5-92f5-026364f734c4 admin admin] The token cache is being invalidate because project 82831792f11f4dcd9770f009448e796e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:13.891842 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-22ebb67e-fcc3-40d5-92f5-026364f734c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:13.893066 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-22ebb67e-fcc3-40d5-92f5-026364f734c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:13.894758 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 866/1762] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:13 2022] DELETE /identity/v3/projects/82831792f11f4dcd9770f009448e796e => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:13.909563 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebb67e-fcc3-40d5-92f5-026364f734c4 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:13.910502 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-22ebb67e-fcc3-40d5-92f5-026364f734c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:13.920428 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:14.122632 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebb67e-fcc3-40d5-92f5-026364f734c4 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:14.157659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1fd96-badb-4ce7-ba3c-e5a731e3e84c admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:14.186945 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2cf3e-5345-43d6-ba7f-cf39db841931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:14.192933 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2cf3e-5345-43d6-ba7f-cf39db841931 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:14.198918 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2cf3e-5345-43d6-ba7f-cf39db841931 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:14.202260 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2cf3e-5345-43d6-ba7f-cf39db841931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:14.227492 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-05a2cf3e-5345-43d6-ba7f-cf39db841931 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:14.256350 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12233cf8-58d9-4705-bca1-e66e8ea924b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:14.257134 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12233cf8-58d9-4705-bca1-e66e8ea924b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:14.257463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12233cf8-58d9-4705-bca1-e66e8ea924b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:14.257877 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12233cf8-58d9-4705-bca1-e66e8ea924b9 admin admin] PATH_INFO: `/v3/projects/a73dce8e992b4df6af7887cfa64e8c5b/users/a03e38fb0fbd40b6a2da1669774f9bb6/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:14.322956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12233cf8-58d9-4705-bca1-e66e8ea924b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a73dce8e992b4df6af7887cfa64e8c5b, user_id=a03e38fb0fbd40b6a2da1669774f9bb6, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:14.325717 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12233cf8-58d9-4705-bca1-e66e8ea924b9 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:14.360496 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 867/1763] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:13 2022] PUT /identity/v3/projects/a73dce8e992b4df6af7887cfa64e8c5b/users/a03e38fb0fbd40b6a2da1669774f9bb6/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:14.361985 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12233cf8-58d9-4705-bca1-e66e8ea924b9 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:14.365915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-12233cf8-58d9-4705-bca1-e66e8ea924b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:14.544164 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:14.584440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2cf3e-5345-43d6-ba7f-cf39db841931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:14.585699 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2cf3e-5345-43d6-ba7f-cf39db841931 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:14.608652 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 897/1764] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 729 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:14.610366 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2cf3e-5345-43d6-ba7f-cf39db841931 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:14.613805 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-05a2cf3e-5345-43d6-ba7f-cf39db841931 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:14.651053 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12233cf8-58d9-4705-bca1-e66e8ea924b9 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:14.671502 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea2dfd-be8a-43b6-8be2-937fb08042a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:14.672359 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea2dfd-be8a-43b6-8be2-937fb08042a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:14.673381 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea2dfd-be8a-43b6-8be2-937fb08042a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:14.673773 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea2dfd-be8a-43b6-8be2-937fb08042a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:14.674690 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea2dfd-be8a-43b6-8be2-937fb08042a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:14.682837 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea2dfd-be8a-43b6-8be2-937fb08042a6 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:14.705976 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2cf3e-5345-43d6-ba7f-cf39db841931 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:14.727560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ad1213-70ae-4c19-99dd-7ada87ffced0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:14.732808 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad1213-70ae-4c19-99dd-7ada87ffced0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:14.733456 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad1213-70ae-4c19-99dd-7ada87ffced0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:14.736530 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad1213-70ae-4c19-99dd-7ada87ffced0 admin admin] PATH_INFO: `/v3/users/642dc48b0b394b25a15c996af3a1637f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:14.745664 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ad1213-70ae-4c19-99dd-7ada87ffced0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=642dc48b0b394b25a15c996af3a1637f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:14.753140 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ad1213-70ae-4c19-99dd-7ada87ffced0 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:14.760288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 868/1765] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:14 2022] POST /identity/v3/users => generated 296 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:14.768765 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea2dfd-be8a-43b6-8be2-937fb08042a6 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:14.777562 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-89ea2dfd-be8a-43b6-8be2-937fb08042a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:14.837915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea2dfd-be8a-43b6-8be2-937fb08042a6 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:14.859866 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e20541-d36e-40b8-b4da-8d33368e2083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:14.861592 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e20541-d36e-40b8-b4da-8d33368e2083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:14.862052 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e20541-d36e-40b8-b4da-8d33368e2083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:14.863308 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e20541-d36e-40b8-b4da-8d33368e2083 admin admin] PATH_INFO: `/v3/projects/c4f0c7a35c014e69bd36bc3dd7021844` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:14.868642 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e20541-d36e-40b8-b4da-8d33368e2083 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4f0c7a35c014e69bd36bc3dd7021844)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:14.871663 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e20541-d36e-40b8-b4da-8d33368e2083 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:14.915072 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e0ad1213-70ae-4c19-99dd-7ada87ffced0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '642dc48b0b394b25a15c996af3a1637f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:14.950647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e0ad1213-70ae-4c19-99dd-7ada87ffced0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '642dc48b0b394b25a15c996af3a1637f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:14.958056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e0ad1213-70ae-4c19-99dd-7ada87ffced0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '642dc48b0b394b25a15c996af3a1637f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:14.962744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e0ad1213-70ae-4c19-99dd-7ada87ffced0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '642dc48b0b394b25a15c996af3a1637f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:15.005590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e0ad1213-70ae-4c19-99dd-7ada87ffced0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '642dc48b0b394b25a15c996af3a1637f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:15.042796 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 898/1766] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:14 2022] DELETE /identity/v3/users/642dc48b0b394b25a15c996af3a1637f => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:15.049830 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ad1213-70ae-4c19-99dd-7ada87ffced0 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:15.066428 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e0ad1213-70ae-4c19-99dd-7ada87ffced0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:15.118106 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-e0e20541-d36e-40b8-b4da-8d33368e2083 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4f0c7a35c014e69bd36bc3dd7021844', 'request_id': 'req-e0e20541-d36e-40b8-b4da-8d33368e2083'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:15.145472 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-e0e20541-d36e-40b8-b4da-8d33368e2083 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4f0c7a35c014e69bd36bc3dd7021844', 'request_id': 'req-e0e20541-d36e-40b8-b4da-8d33368e2083'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:15.161247 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:15.176896 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-e0e20541-d36e-40b8-b4da-8d33368e2083 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4f0c7a35c014e69bd36bc3dd7021844', 'request_id': 'req-e0e20541-d36e-40b8-b4da-8d33368e2083'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:15.237681 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-e0e20541-d36e-40b8-b4da-8d33368e2083 admin admin] The token cache is being invalidate because project c4f0c7a35c014e69bd36bc3dd7021844 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:15.243746 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-e0e20541-d36e-40b8-b4da-8d33368e2083 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:15.245299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-e0e20541-d36e-40b8-b4da-8d33368e2083 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:15.253819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 869/1767] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:14 2022] DELETE /identity/v3/projects/c4f0c7a35c014e69bd36bc3dd7021844 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:15.261579 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e20541-d36e-40b8-b4da-8d33368e2083 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:15.265927 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-e0e20541-d36e-40b8-b4da-8d33368e2083 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:15.358345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ad1213-70ae-4c19-99dd-7ada87ffced0 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:15.398314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f628e6e-d39e-4471-a70b-eba92ba0e3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:15.402848 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f628e6e-d39e-4471-a70b-eba92ba0e3aa admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:15.412526 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f628e6e-d39e-4471-a70b-eba92ba0e3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:15.413756 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f628e6e-d39e-4471-a70b-eba92ba0e3aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:15.418860 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f628e6e-d39e-4471-a70b-eba92ba0e3aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:15.429590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f628e6e-d39e-4471-a70b-eba92ba0e3aa admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:15.469147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e20541-d36e-40b8-b4da-8d33368e2083 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:15.485010 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a1157-eb39-466d-8157-7e039c4edff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:15.486063 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a1157-eb39-466d-8157-7e039c4edff0 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:15.486336 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a1157-eb39-466d-8157-7e039c4edff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:15.487439 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a1157-eb39-466d-8157-7e039c4edff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:15.487439 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a1157-eb39-466d-8157-7e039c4edff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:15.489023 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a1157-eb39-466d-8157-7e039c4edff0 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:15.522672 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 870/1768] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:15.524478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0a59e-88fb-4498-b6ae-38319d820e5a None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:15.524772 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0a59e-88fb-4498-b6ae-38319d820e5a None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:15.525052 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0a59e-88fb-4498-b6ae-38319d820e5a None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:15.532238 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 871/1769] 10.222.0.16 () {54 vars in 921 bytes} [Mon Sep 5 12:19:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:19:15.533315 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b0a59e-88fb-4498-b6ae-38319d820e5a None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:15.541504 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-d6b0a59e-88fb-4498-b6ae-38319d820e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:15.559215 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 899/1770] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 514 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:15.561231 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f628e6e-d39e-4471-a70b-eba92ba0e3aa admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:15.576332 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b0a59e-88fb-4498-b6ae-38319d820e5a None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:15.588863 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b5305-c3fd-4d9b-99a8-fefe968dd8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:15.589706 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b5305-c3fd-4d9b-99a8-fefe968dd8f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:15.590254 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b5305-c3fd-4d9b-99a8-fefe968dd8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:15.590708 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b5305-c3fd-4d9b-99a8-fefe968dd8f1 admin admin] PATH_INFO: `/v3/users/2d14bc0b2a7a487facbebdb843872e74` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:15.593941 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b5305-c3fd-4d9b-99a8-fefe968dd8f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d14bc0b2a7a487facbebdb843872e74)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:15.596284 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b5305-c3fd-4d9b-99a8-fefe968dd8f1 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:15.658392 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f628e6e-d39e-4471-a70b-eba92ba0e3aa admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:15.685643 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-e07b5305-c3fd-4d9b-99a8-fefe968dd8f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d14bc0b2a7a487facbebdb843872e74'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:15.692794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=81,keystone:INSERT=5,keystone:DELETE=14 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:15.698333 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102a381-c0d5-4d23-b532-aeb2329e70c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:15.705280 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-e07b5305-c3fd-4d9b-99a8-fefe968dd8f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d14bc0b2a7a487facbebdb843872e74'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:15.715794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-e07b5305-c3fd-4d9b-99a8-fefe968dd8f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d14bc0b2a7a487facbebdb843872e74'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:15.718461 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102a381-c0d5-4d23-b532-aeb2329e70c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:15.720347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102a381-c0d5-4d23-b532-aeb2329e70c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:15.721489 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102a381-c0d5-4d23-b532-aeb2329e70c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:15.725184 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1102a381-c0d5-4d23-b532-aeb2329e70c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:15.731274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1102a381-c0d5-4d23-b532-aeb2329e70c4 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:15.784294 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 900/1771] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:15.787994 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102a381-c0d5-4d23-b532-aeb2329e70c4 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:15.793924 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1102a381-c0d5-4d23-b532-aeb2329e70c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:15.848393 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-e07b5305-c3fd-4d9b-99a8-fefe968dd8f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d14bc0b2a7a487facbebdb843872e74'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:15.853242 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102a381-c0d5-4d23-b532-aeb2329e70c4 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:15.880850 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-e07b5305-c3fd-4d9b-99a8-fefe968dd8f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d14bc0b2a7a487facbebdb843872e74'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:15.893119 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f57e2-c6c9-47b5-af69-a8befbe9e67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:15.895957 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f57e2-c6c9-47b5-af69-a8befbe9e67d admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:15.897795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f57e2-c6c9-47b5-af69-a8befbe9e67d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:15.900791 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f57e2-c6c9-47b5-af69-a8befbe9e67d admin admin] PATH_INFO: `/v3/projects/a73dce8e992b4df6af7887cfa64e8c5b/users/a03e38fb0fbd40b6a2da1669774f9bb6/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:15.910529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f57e2-c6c9-47b5-af69-a8befbe9e67d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a73dce8e992b4df6af7887cfa64e8c5b, user_id=a03e38fb0fbd40b6a2da1669774f9bb6, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:15.912975 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 872/1772] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:15 2022] DELETE /identity/v3/users/2d14bc0b2a7a487facbebdb843872e74 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:15.922222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f57e2-c6c9-47b5-af69-a8befbe9e67d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:15.923108 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f691e-0b45-4376-b68b-f0a1028a9c26 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:15.924215 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f691e-0b45-4376-b68b-f0a1028a9c26 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:15.925108 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f691e-0b45-4376-b68b-f0a1028a9c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:15.993147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-bc0f691e-0b45-4376-b68b-f0a1028a9c26 None None] MFA Rules not processed for user `7cda290418504f0db85d665479a05353`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:15.996241 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-bc0f691e-0b45-4376-b68b-f0a1028a9c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:15.998182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 901/1773] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:15 2022] PUT /identity/v3/projects/a73dce8e992b4df6af7887cfa64e8c5b/users/a03e38fb0fbd40b6a2da1669774f9bb6/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:16.002696 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f57e2-c6c9-47b5-af69-a8befbe9e67d admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:16.019110 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d48f57e2-c6c9-47b5-af69-a8befbe9e67d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:16.195102 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f57e2-c6c9-47b5-af69-a8befbe9e67d admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:16.200727 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a665c64c-d8f7-498a-a3f1-675c3070b512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:16.201719 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665c64c-d8f7-498a-a3f1-675c3070b512 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:16.202537 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665c64c-d8f7-498a-a3f1-675c3070b512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:16.203055 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665c64c-d8f7-498a-a3f1-675c3070b512 admin admin] PATH_INFO: `/v3/projects/48b1cbb8b8504c1da4fc4a54ede3a549/users/bfacaab427ac4562bae47fe50ea909c3/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:16.237098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a665c64c-d8f7-498a-a3f1-675c3070b512 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48b1cbb8b8504c1da4fc4a54ede3a549, user_id=bfacaab427ac4562bae47fe50ea909c3, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:16.245330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a665c64c-d8f7-498a-a3f1-675c3070b512 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:16.281489 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 902/1774] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:16 2022] PUT /identity/v3/projects/48b1cbb8b8504c1da4fc4a54ede3a549/users/bfacaab427ac4562bae47fe50ea909c3/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:16.283680 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf89b9-6a81-4514-bfe4-d0e0cfb700a8 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:16.284407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf89b9-6a81-4514-bfe4-d0e0cfb700a8 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:16.286210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf89b9-6a81-4514-bfe4-d0e0cfb700a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:16.328534 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 873/1775] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Sep 5 12:19:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:16.330943 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae914d-3877-4e83-a61c-12a749279441 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:16.331791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae914d-3877-4e83-a61c-12a749279441 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:16.332155 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae914d-3877-4e83-a61c-12a749279441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:16.400854 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-1cdf89b9-6a81-4514-bfe4-d0e0cfb700a8 None None] MFA Rules not processed for user `a03e38fb0fbd40b6a2da1669774f9bb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:16.404404 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1cdf89b9-6a81-4514-bfe4-d0e0cfb700a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:16.475725 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-78ae914d-3877-4e83-a61c-12a749279441 None None] MFA Rules not processed for user `bfacaab427ac4562bae47fe50ea909c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:16.478368 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-78ae914d-3877-4e83-a61c-12a749279441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:16.808558 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 874/1776] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:16 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 479 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:16.810267 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae914d-3877-4e83-a61c-12a749279441 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:16.812752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-78ae914d-3877-4e83-a61c-12a749279441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:16.825702 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 903/1777] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:16 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 543 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:16.828677 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdf89b9-6a81-4514-bfe4-d0e0cfb700a8 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:16.842036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:17.001100 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:17.118817 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdf89b9-6a81-4514-bfe4-d0e0cfb700a8 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:17.141869 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae914d-3877-4e83-a61c-12a749279441 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:17.142773 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121f195-a2a5-4837-8e46-27b76a474561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:17.143604 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121f195-a2a5-4837-8e46-27b76a474561 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:17.144032 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121f195-a2a5-4837-8e46-27b76a474561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:17.144404 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121f195-a2a5-4837-8e46-27b76a474561 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:17.153125 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d121f195-a2a5-4837-8e46-27b76a474561 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:17.160024 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d121f195-a2a5-4837-8e46-27b76a474561 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:17.178102 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81dd74-ac00-43ae-b6b6-295ae3ff425b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:17.181928 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81dd74-ac00-43ae-b6b6-295ae3ff425b service nova] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:17.182305 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81dd74-ac00-43ae-b6b6-295ae3ff425b service nova] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:17.182768 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81dd74-ac00-43ae-b6b6-295ae3ff425b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:17.224509 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81dd74-ac00-43ae-b6b6-295ae3ff425b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:17.234991 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81dd74-ac00-43ae-b6b6-295ae3ff425b service nova] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:17.235299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 904/1778] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:16 2022] POST /identity/v3/projects => generated 393 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:17.239076 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121f195-a2a5-4837-8e46-27b76a474561 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:17.245555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d121f195-a2a5-4837-8e46-27b76a474561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:17.300131 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 875/1779] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:17.301004 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81dd74-ac00-43ae-b6b6-295ae3ff425b service nova] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:17.386820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81dd74-ac00-43ae-b6b6-295ae3ff425b service nova] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:17.388893 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121f195-a2a5-4837-8e46-27b76a474561 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:17.398043 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dba047-0029-4d74-bd11-52dbf942f298 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:17.398474 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dba047-0029-4d74-bd11-52dbf942f298 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:17.398759 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dba047-0029-4d74-bd11-52dbf942f298 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:17.399332 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dba047-0029-4d74-bd11-52dbf942f298 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:17.576160 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529ad7b-4fc9-42d4-95fa-e1c51bed85c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:17.578102 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529ad7b-4fc9-42d4-95fa-e1c51bed85c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:17.578568 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529ad7b-4fc9-42d4-95fa-e1c51bed85c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:17.579001 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529ad7b-4fc9-42d4-95fa-e1c51bed85c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:17.581745 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a529ad7b-4fc9-42d4-95fa-e1c51bed85c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:17.584107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a529ad7b-4fc9-42d4-95fa-e1c51bed85c8 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:17.617256 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dba047-0029-4d74-bd11-52dbf942f298 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:17.618915 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dba047-0029-4d74-bd11-52dbf942f298 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:17.659305 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 876/1780] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:17 2022] POST /identity/v3/users => generated 303 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:17.661313 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 905/1781] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:17 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:17.663270 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dba047-0029-4d74-bd11-52dbf942f298 service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:17.663925 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10640d7f-9d14-40aa-9ac5-b41bce6fdd94 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:17.664390 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10640d7f-9d14-40aa-9ac5-b41bce6fdd94 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:17.664816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10640d7f-9d14-40aa-9ac5-b41bce6fdd94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:17.694599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dba047-0029-4d74-bd11-52dbf942f298 service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:17.722417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-10640d7f-9d14-40aa-9ac5-b41bce6fdd94 None None] MFA Rules not processed for user `45bd61574c6c4ebb9d6fe0c87228065f`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:17.725244 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-10640d7f-9d14-40aa-9ac5-b41bce6fdd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:17.728147 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c955c7f-1755-4ace-aeea-4d6af1a0954a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:17.732963 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c955c7f-1755-4ace-aeea-4d6af1a0954a admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:17.732963 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c955c7f-1755-4ace-aeea-4d6af1a0954a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:17.732963 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c955c7f-1755-4ace-aeea-4d6af1a0954a admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:17.737294 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c955c7f-1755-4ace-aeea-4d6af1a0954a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:17.745606 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c955c7f-1755-4ace-aeea-4d6af1a0954a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:17.805931 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 906/1782] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:17.808976 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c955c7f-1755-4ace-aeea-4d6af1a0954a admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:17.817518 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-0c955c7f-1755-4ace-aeea-4d6af1a0954a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:17.854677 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c955c7f-1755-4ace-aeea-4d6af1a0954a admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:17.869016 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291bc4cf-ddb4-4aa5-bdcd-3c2e3f745a17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:17.869694 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291bc4cf-ddb4-4aa5-bdcd-3c2e3f745a17 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:17.870000 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291bc4cf-ddb4-4aa5-bdcd-3c2e3f745a17 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:17.870278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291bc4cf-ddb4-4aa5-bdcd-3c2e3f745a17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:17.874980 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-291bc4cf-ddb4-4aa5-bdcd-3c2e3f745a17 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:17.902647 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 877/1783] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:17.904161 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10640d7f-9d14-40aa-9ac5-b41bce6fdd94 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:17.972659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10640d7f-9d14-40aa-9ac5-b41bce6fdd94 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:18.000833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb22414-a5d3-4532-a38a-ce8a87da9560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:18.006256 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb22414-a5d3-4532-a38a-ce8a87da9560 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:18.006566 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb22414-a5d3-4532-a38a-ce8a87da9560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:18.007232 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb22414-a5d3-4532-a38a-ce8a87da9560 admin admin] PATH_INFO: `/v3/projects/7fa438bda19443c5bcd640af2b89736f/users/5879ec142a6a46df8eec467a70a5be33/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:18.018673 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:18.056797 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb22414-a5d3-4532-a38a-ce8a87da9560 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fa438bda19443c5bcd640af2b89736f, user_id=5879ec142a6a46df8eec467a70a5be33, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:18.059868 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb22414-a5d3-4532-a38a-ce8a87da9560 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:18.109719 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 878/1784] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:17 2022] PUT /identity/v3/projects/7fa438bda19443c5bcd640af2b89736f/users/5879ec142a6a46df8eec467a70a5be33/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:18.114419 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb22414-a5d3-4532-a38a-ce8a87da9560 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:18.172747 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291bc4cf-ddb4-4aa5-bdcd-3c2e3f745a17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:18.175791 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291bc4cf-ddb4-4aa5-bdcd-3c2e3f745a17 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:18.272107 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 907/1785] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:18.281826 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:18.319904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb22414-a5d3-4532-a38a-ce8a87da9560 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:18.329499 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac14b2b-df43-4e56-bda9-5f2145556e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:18.330639 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac14b2b-df43-4e56-bda9-5f2145556e5e admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:18.330947 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac14b2b-df43-4e56-bda9-5f2145556e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:18.332059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac14b2b-df43-4e56-bda9-5f2145556e5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:18.333276 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac14b2b-df43-4e56-bda9-5f2145556e5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:18.336575 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac14b2b-df43-4e56-bda9-5f2145556e5e admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:18.362329 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 879/1786] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:18 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:18.362561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291bc4cf-ddb4-4aa5-bdcd-3c2e3f745a17 service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:18.401598 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291bc4cf-ddb4-4aa5-bdcd-3c2e3f745a17 service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:18.416557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822a13e-ebcc-446e-a980-f650b8afda4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:18.418374 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822a13e-ebcc-446e-a980-f650b8afda4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:18.419120 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822a13e-ebcc-446e-a980-f650b8afda4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:18.420126 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822a13e-ebcc-446e-a980-f650b8afda4b admin admin] PATH_INFO: `/v3/projects/7fa438bda19443c5bcd640af2b89736f/users/5879ec142a6a46df8eec467a70a5be33/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:18.431936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d822a13e-ebcc-446e-a980-f650b8afda4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fa438bda19443c5bcd640af2b89736f, user_id=5879ec142a6a46df8eec467a70a5be33, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:18.439132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d822a13e-ebcc-446e-a980-f650b8afda4b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:18.515304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 908/1787] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:18 2022] PUT /identity/v3/projects/7fa438bda19443c5bcd640af2b89736f/users/5879ec142a6a46df8eec467a70a5be33/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:19:18.541286 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4a63c-93de-447c-b88a-bf8dc6856401 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:18.541576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4a63c-93de-447c-b88a-bf8dc6856401 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:18.541861 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4a63c-93de-447c-b88a-bf8dc6856401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:18.631996 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-ade4a63c-93de-447c-b88a-bf8dc6856401 None None] MFA Rules not processed for user `5879ec142a6a46df8eec467a70a5be33`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:18.639181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-ade4a63c-93de-447c-b88a-bf8dc6856401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:18.904041 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 880/1788] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:18 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:18.918070 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822a13e-ebcc-446e-a980-f650b8afda4b admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:19.112790 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822a13e-ebcc-446e-a980-f650b8afda4b admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:19.126766 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f9108-bb1f-4bff-8c8f-e84ab792fb7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:19.130300 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f9108-bb1f-4bff-8c8f-e84ab792fb7a service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:19.131613 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f9108-bb1f-4bff-8c8f-e84ab792fb7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:19.133531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f9108-bb1f-4bff-8c8f-e84ab792fb7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:19.174671 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f9108-bb1f-4bff-8c8f-e84ab792fb7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:19.188552 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f9108-bb1f-4bff-8c8f-e84ab792fb7a service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:19.254152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 909/1789] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:18 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:19.446023 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4a63c-93de-447c-b88a-bf8dc6856401 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:19.471479 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4a63c-93de-447c-b88a-bf8dc6856401 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:19.489026 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f81bc-e1ea-45cb-94f2-46ac6c9d926d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:19.494289 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f81bc-e1ea-45cb-94f2-46ac6c9d926d service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:19.494946 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f81bc-e1ea-45cb-94f2-46ac6c9d926d service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:19.497289 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f81bc-e1ea-45cb-94f2-46ac6c9d926d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:19.674511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f81bc-e1ea-45cb-94f2-46ac6c9d926d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:19.677434 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f81bc-e1ea-45cb-94f2-46ac6c9d926d service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:19.692426 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 881/1790] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:19 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:21.084890 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f9108-bb1f-4bff-8c8f-e84ab792fb7a service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:21.088156 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-972f9108-bb1f-4bff-8c8f-e84ab792fb7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:21.381386 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f9108-bb1f-4bff-8c8f-e84ab792fb7a service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:21.424368 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858aee1e-8b6c-4347-b824-edad1016dfe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:21.425417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858aee1e-8b6c-4347-b824-edad1016dfe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:21.425844 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858aee1e-8b6c-4347-b824-edad1016dfe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:21.426338 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858aee1e-8b6c-4347-b824-edad1016dfe4 admin admin] PATH_INFO: `/v3/projects/7efc6841b58740b8b56fd5b50dccfd19` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:21.434476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858aee1e-8b6c-4347-b824-edad1016dfe4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7efc6841b58740b8b56fd5b50dccfd19)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:21.438347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858aee1e-8b6c-4347-b824-edad1016dfe4 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:21.609132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f81bc-e1ea-45cb-94f2-46ac6c9d926d service cinder] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:21.614188 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-4b1f81bc-e1ea-45cb-94f2-46ac6c9d926d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:21.706880 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-858aee1e-8b6c-4347-b824-edad1016dfe4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7efc6841b58740b8b56fd5b50dccfd19', 'request_id': 'req-858aee1e-8b6c-4347-b824-edad1016dfe4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:21.724657 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f81bc-e1ea-45cb-94f2-46ac6c9d926d service cinder] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:21.777679 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-858aee1e-8b6c-4347-b824-edad1016dfe4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7efc6841b58740b8b56fd5b50dccfd19', 'request_id': 'req-858aee1e-8b6c-4347-b824-edad1016dfe4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:21.860185 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-858aee1e-8b6c-4347-b824-edad1016dfe4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7efc6841b58740b8b56fd5b50dccfd19', 'request_id': 'req-858aee1e-8b6c-4347-b824-edad1016dfe4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:21.884441 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:21.957210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-858aee1e-8b6c-4347-b824-edad1016dfe4 admin admin] The token cache is being invalidate because project 7efc6841b58740b8b56fd5b50dccfd19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:21.958379 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-858aee1e-8b6c-4347-b824-edad1016dfe4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:21.963254 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-858aee1e-8b6c-4347-b824-edad1016dfe4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:22.010789 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 910/1791] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:21 2022] DELETE /identity/v3/projects/7efc6841b58740b8b56fd5b50dccfd19 => generated 0 bytes in 923 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:22.013590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858aee1e-8b6c-4347-b824-edad1016dfe4 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:22.034305 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-858aee1e-8b6c-4347-b824-edad1016dfe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:22.184178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1da2ab-ce0d-4955-ad26-e158e1edc018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:22.185266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1da2ab-ce0d-4955-ad26-e158e1edc018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:22.185883 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1da2ab-ce0d-4955-ad26-e158e1edc018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:22.186596 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1da2ab-ce0d-4955-ad26-e158e1edc018 admin admin] PATH_INFO: `/v3/projects/1774836db0fa4ebebad29925812451dc` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:22.190421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1da2ab-ce0d-4955-ad26-e158e1edc018 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1774836db0fa4ebebad29925812451dc)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:22.197708 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1da2ab-ce0d-4955-ad26-e158e1edc018 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:22.218620 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:22.260427 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858aee1e-8b6c-4347-b824-edad1016dfe4 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:22.300197 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80771170-59bb-40a3-821b-ed0947d2cac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:22.301221 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80771170-59bb-40a3-821b-ed0947d2cac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:22.301736 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80771170-59bb-40a3-821b-ed0947d2cac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:22.302211 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80771170-59bb-40a3-821b-ed0947d2cac5 admin admin] PATH_INFO: `/v3/projects/c87b2c6b54de4dcfa8dd11e47e4466d1` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:22.305394 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80771170-59bb-40a3-821b-ed0947d2cac5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c87b2c6b54de4dcfa8dd11e47e4466d1)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:22.309073 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80771170-59bb-40a3-821b-ed0947d2cac5 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:22.347787 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3a1da2ab-ce0d-4955-ad26-e158e1edc018 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1774836db0fa4ebebad29925812451dc', 'request_id': 'req-3a1da2ab-ce0d-4955-ad26-e158e1edc018'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:22.364248 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3a1da2ab-ce0d-4955-ad26-e158e1edc018 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1774836db0fa4ebebad29925812451dc', 'request_id': 'req-3a1da2ab-ce0d-4955-ad26-e158e1edc018'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:22.392858 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3a1da2ab-ce0d-4955-ad26-e158e1edc018 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1774836db0fa4ebebad29925812451dc', 'request_id': 'req-3a1da2ab-ce0d-4955-ad26-e158e1edc018'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:22.413314 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3a1da2ab-ce0d-4955-ad26-e158e1edc018 admin admin] The token cache is being invalidate because project 1774836db0fa4ebebad29925812451dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:22.413613 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3a1da2ab-ce0d-4955-ad26-e158e1edc018 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:22.415242 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3a1da2ab-ce0d-4955-ad26-e158e1edc018 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:22.417652 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 882/1792] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:21 2022] DELETE /identity/v3/projects/1774836db0fa4ebebad29925812451dc => generated 0 bytes in 810 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:19:22.432880 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b1b41-b072-490c-a9f3-bddec0383525 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:22.433199 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b1b41-b072-490c-a9f3-bddec0383525 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:22.433481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b1b41-b072-490c-a9f3-bddec0383525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:22.481972 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-80771170-59bb-40a3-821b-ed0947d2cac5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c87b2c6b54de4dcfa8dd11e47e4466d1', 'request_id': 'req-80771170-59bb-40a3-821b-ed0947d2cac5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:22.512160 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-80771170-59bb-40a3-821b-ed0947d2cac5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c87b2c6b54de4dcfa8dd11e47e4466d1', 'request_id': 'req-80771170-59bb-40a3-821b-ed0947d2cac5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:22.528081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-80771170-59bb-40a3-821b-ed0947d2cac5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c87b2c6b54de4dcfa8dd11e47e4466d1', 'request_id': 'req-80771170-59bb-40a3-821b-ed0947d2cac5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:22.556109 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-137b1b41-b072-490c-a9f3-bddec0383525 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:22.558316 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-137b1b41-b072-490c-a9f3-bddec0383525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:22.562445 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-80771170-59bb-40a3-821b-ed0947d2cac5 admin admin] The token cache is being invalidate because project c87b2c6b54de4dcfa8dd11e47e4466d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:22.564019 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-80771170-59bb-40a3-821b-ed0947d2cac5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:22.565371 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-80771170-59bb-40a3-821b-ed0947d2cac5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:22.574345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 911/1793] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:22 2022] DELETE /identity/v3/projects/c87b2c6b54de4dcfa8dd11e47e4466d1 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:22.577272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd71a1-f68a-43e6-8647-1b4ae3f2ef8b None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:22.577864 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd71a1-f68a-43e6-8647-1b4ae3f2ef8b None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:22.578153 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd71a1-f68a-43e6-8647-1b4ae3f2ef8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:22.636299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-02dd71a1-f68a-43e6-8647-1b4ae3f2ef8b None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:22.637588 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-02dd71a1-f68a-43e6-8647-1b4ae3f2ef8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:22.719476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 883/1794] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:22.722188 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b60553-a390-4cc6-bb53-ffb2007d7e61 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:22.722764 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b60553-a390-4cc6-bb53-ffb2007d7e61 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:22.723945 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b60553-a390-4cc6-bb53-ffb2007d7e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:22.744634 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 912/1795] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:22.745978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766c373-89ef-408b-8e97-38e0bf758198 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:22.749725 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766c373-89ef-408b-8e97-38e0bf758198 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:22.749725 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766c373-89ef-408b-8e97-38e0bf758198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:22.779317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-32b60553-a390-4cc6-bb53-ffb2007d7e61 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:22.783584 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-32b60553-a390-4cc6-bb53-ffb2007d7e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:22.793341 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 884/1796] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:22.804971 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae334a-c38c-4e2d-a7ca-1c30f565a349 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:22.805578 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae334a-c38c-4e2d-a7ca-1c30f565a349 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:22.806503 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae334a-c38c-4e2d-a7ca-1c30f565a349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:22.833634 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-b766c373-89ef-408b-8e97-38e0bf758198 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:22.837099 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b766c373-89ef-408b-8e97-38e0bf758198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:22.841808 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-f1ae334a-c38c-4e2d-a7ca-1c30f565a349 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:22.843348 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-f1ae334a-c38c-4e2d-a7ca-1c30f565a349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:22.862905 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 885/1797] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:22.862905 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ed7eb-2232-4f21-bd74-59ef0e7c96da None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:22.862905 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ed7eb-2232-4f21-bd74-59ef0e7c96da None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:22.862905 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ed7eb-2232-4f21-bd74-59ef0e7c96da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:22.883389 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 913/1798] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:22.883389 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766c373-89ef-408b-8e97-38e0bf758198 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:22.896764 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-855ed7eb-2232-4f21-bd74-59ef0e7c96da None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:22.903500 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-855ed7eb-2232-4f21-bd74-59ef0e7c96da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:22.919371 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 886/1799] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:22.921987 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ed7eb-2232-4f21-bd74-59ef0e7c96da None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:22.929769 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766c373-89ef-408b-8e97-38e0bf758198 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:22.937614 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ed7eb-2232-4f21-bd74-59ef0e7c96da None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:22.943970 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfebb52-5143-4646-b228-ad20029e01d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:22.945218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfebb52-5143-4646-b228-ad20029e01d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:22.945728 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfebb52-5143-4646-b228-ad20029e01d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:22.946292 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfebb52-5143-4646-b228-ad20029e01d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:22.948322 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfebb52-5143-4646-b228-ad20029e01d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:19:22.949962 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfebb52-5143-4646-b228-ad20029e01d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:22.950566 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2185f7-5aa0-4db5-9d4f-daa0dae82208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:22.952014 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2185f7-5aa0-4db5-9d4f-daa0dae82208 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:22.952213 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfebb52-5143-4646-b228-ad20029e01d5 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:22.954049 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2185f7-5aa0-4db5-9d4f-daa0dae82208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:22.955434 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2185f7-5aa0-4db5-9d4f-daa0dae82208 admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:22.958772 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2185f7-5aa0-4db5-9d4f-daa0dae82208 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:19:22.960039 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2185f7-5aa0-4db5-9d4f-daa0dae82208 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:22.963987 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2185f7-5aa0-4db5-9d4f-daa0dae82208 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:22.973978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 914/1800] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:19:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:19:22.974419 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfebb52-5143-4646-b228-ad20029e01d5 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:22.988867 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:23.002891 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfebb52-5143-4646-b228-ad20029e01d5 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:23.017465 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36067e70-ae65-4bf5-884f-181ad3a29386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:23.017465 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36067e70-ae65-4bf5-884f-181ad3a29386 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:23.017465 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36067e70-ae65-4bf5-884f-181ad3a29386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:23.017465 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36067e70-ae65-4bf5-884f-181ad3a29386 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:23.018288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 887/1801] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:19:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:19:23.018437 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36067e70-ae65-4bf5-884f-181ad3a29386 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:19:23.018437 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36067e70-ae65-4bf5-884f-181ad3a29386 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:23.018744 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef95a9-81a1-427d-bed2-ac79df1cdf52 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:23.018744 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef95a9-81a1-427d-bed2-ac79df1cdf52 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:23.018744 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef95a9-81a1-427d-bed2-ac79df1cdf52 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:23.019104 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36067e70-ae65-4bf5-884f-181ad3a29386 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:23.022285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 888/1802] 10.222.0.16 () {54 vars in 921 bytes} [Mon Sep 5 12:19:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:19:23.023494 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fef95a9-81a1-427d-bed2-ac79df1cdf52 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:23.049226 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 915/1803] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:19:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:19:23.052291 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36067e70-ae65-4bf5-884f-181ad3a29386 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:23.076457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36067e70-ae65-4bf5-884f-181ad3a29386 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:23.084981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509a84a-a961-4c1f-980c-59d4e137273b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:23.086551 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509a84a-a961-4c1f-980c-59d4e137273b admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:23.087058 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509a84a-a961-4c1f-980c-59d4e137273b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:23.087566 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509a84a-a961-4c1f-980c-59d4e137273b admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:23.088587 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509a84a-a961-4c1f-980c-59d4e137273b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:23.091636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fef95a9-81a1-427d-bed2-ac79df1cdf52 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:23.092212 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509a84a-a961-4c1f-980c-59d4e137273b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:23.103033 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8bf62e-6a48-4143-95af-00c56b90f8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:23.103716 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8bf62e-6a48-4143-95af-00c56b90f8ac admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:23.104001 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8bf62e-6a48-4143-95af-00c56b90f8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:23.104273 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8bf62e-6a48-4143-95af-00c56b90f8ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:23.105527 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8bf62e-6a48-4143-95af-00c56b90f8ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:23.107581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8bf62e-6a48-4143-95af-00c56b90f8ac admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:23.173041 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 889/1804] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:23 2022] POST /identity/v3/projects => generated 399 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:23.173266 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 916/1805] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:23 2022] POST /identity/v3/projects => generated 355 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:23.173356 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483954b-66ef-4615-93bd-b05c5e7beb74 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:23.173356 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483954b-66ef-4615-93bd-b05c5e7beb74 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:23.173356 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483954b-66ef-4615-93bd-b05c5e7beb74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:23.174004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509a84a-a961-4c1f-980c-59d4e137273b admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:23.209343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-c483954b-66ef-4615-93bd-b05c5e7beb74 None None] MFA Rules not processed for user `7cda290418504f0db85d665479a05353`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:23.215717 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-c483954b-66ef-4615-93bd-b05c5e7beb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:23.295920 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:23.349594 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 890/1806] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Sep 5 12:19:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:23.356729 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c483954b-66ef-4615-93bd-b05c5e7beb74 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:23.357392 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509a84a-a961-4c1f-980c-59d4e137273b admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:23.359190 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-c483954b-66ef-4615-93bd-b05c5e7beb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:23.470766 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c483954b-66ef-4615-93bd-b05c5e7beb74 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:23.478624 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e65e5-7718-433c-973e-6e589bc47c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:23.479854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e65e5-7718-433c-973e-6e589bc47c5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:23.480337 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e65e5-7718-433c-973e-6e589bc47c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:23.480802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e65e5-7718-433c-973e-6e589bc47c5f admin admin] PATH_INFO: `/v3/users/8a50f9ac6fda4544980bcd1009ee80a9` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:23.482838 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e65e5-7718-433c-973e-6e589bc47c5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a50f9ac6fda4544980bcd1009ee80a9)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:23.483081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7311d38-3a7f-4065-8bf1-58628a50887a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:23.484253 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7311d38-3a7f-4065-8bf1-58628a50887a admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:23.484790 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7311d38-3a7f-4065-8bf1-58628a50887a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:23.486577 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7311d38-3a7f-4065-8bf1-58628a50887a admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:23.486577 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7311d38-3a7f-4065-8bf1-58628a50887a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:23.486947 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e65e5-7718-433c-973e-6e589bc47c5f admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:23.490299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7311d38-3a7f-4065-8bf1-58628a50887a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:23.541754 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 917/1807] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:23 2022] POST /identity/v3/projects => generated 369 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:23.543084 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7311d38-3a7f-4065-8bf1-58628a50887a admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:23.570046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-729e65e5-7718-433c-973e-6e589bc47c5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a50f9ac6fda4544980bcd1009ee80a9'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:23.582547 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-729e65e5-7718-433c-973e-6e589bc47c5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a50f9ac6fda4544980bcd1009ee80a9'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:23.583611 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-729e65e5-7718-433c-973e-6e589bc47c5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a50f9ac6fda4544980bcd1009ee80a9'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:23.603290 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-729e65e5-7718-433c-973e-6e589bc47c5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a50f9ac6fda4544980bcd1009ee80a9'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:23.614441 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-729e65e5-7718-433c-973e-6e589bc47c5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a50f9ac6fda4544980bcd1009ee80a9'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:23.622218 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 891/1808] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:23 2022] DELETE /identity/v3/users/8a50f9ac6fda4544980bcd1009ee80a9 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:23.623083 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e65e5-7718-433c-973e-6e589bc47c5f admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:23.628256 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-729e65e5-7718-433c-973e-6e589bc47c5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:23.642735 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7311d38-3a7f-4065-8bf1-58628a50887a admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:23.649886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:23.725655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:23.743426 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e65e5-7718-433c-973e-6e589bc47c5f admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:23.752413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113fd809-0531-4308-a89e-8246adcd1ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:23.753114 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113fd809-0531-4308-a89e-8246adcd1ed7 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:23.753411 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113fd809-0531-4308-a89e-8246adcd1ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:23.754031 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113fd809-0531-4308-a89e-8246adcd1ed7 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:23.755294 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113fd809-0531-4308-a89e-8246adcd1ed7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:23.757786 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113fd809-0531-4308-a89e-8246adcd1ed7 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:23.782314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932da975-9398-4647-9181-0bdd020e243a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:23.783816 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932da975-9398-4647-9181-0bdd020e243a admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:23.784616 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932da975-9398-4647-9181-0bdd020e243a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:23.785903 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932da975-9398-4647-9181-0bdd020e243a admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:23.786237 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 892/1809] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:23 2022] POST /identity/v3/users => generated 300 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:23.786537 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932da975-9398-4647-9181-0bdd020e243a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:23.787242 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113fd809-0531-4308-a89e-8246adcd1ed7 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:23.789001 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-113fd809-0531-4308-a89e-8246adcd1ed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:23.790110 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932da975-9398-4647-9181-0bdd020e243a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:23.838518 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 918/1810] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:23 2022] POST /identity/v3/users => generated 278 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:23.840015 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932da975-9398-4647-9181-0bdd020e243a admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:23.845101 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-932da975-9398-4647-9181-0bdd020e243a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:23.861847 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932da975-9398-4647-9181-0bdd020e243a admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:23.868231 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ee37c-c676-4f24-ad3b-71e81995d1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:23.868881 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ee37c-c676-4f24-ad3b-71e81995d1fc admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:23.869264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ee37c-c676-4f24-ad3b-71e81995d1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:23.869546 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ee37c-c676-4f24-ad3b-71e81995d1fc admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:23.870211 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ee37c-c676-4f24-ad3b-71e81995d1fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:23.872397 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ee37c-c676-4f24-ad3b-71e81995d1fc admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:23.931752 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 919/1811] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:23 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:23.933590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e82b5-1360-43eb-8a3d-8b59c21da820 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:23.935183 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e82b5-1360-43eb-8a3d-8b59c21da820 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:23.935484 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e82b5-1360-43eb-8a3d-8b59c21da820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:23.942221 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113fd809-0531-4308-a89e-8246adcd1ed7 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:23.948546 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb5671-3d1c-4b5d-932e-033a2e6943ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:23.949216 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb5671-3d1c-4b5d-932e-033a2e6943ea service nova] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:23.949513 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb5671-3d1c-4b5d-932e-033a2e6943ea service nova] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:23.949741 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb5671-3d1c-4b5d-932e-033a2e6943ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:23.951994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-77cb5671-3d1c-4b5d-932e-033a2e6943ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:23.967467 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb5671-3d1c-4b5d-932e-033a2e6943ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:23.969650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb5671-3d1c-4b5d-932e-033a2e6943ea service nova] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:23.989287 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 893/1812] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:23.992489 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb5671-3d1c-4b5d-932e-033a2e6943ea service nova] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:24.004951 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-3d6e82b5-1360-43eb-8a3d-8b59c21da820 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:24.007750 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3d6e82b5-1360-43eb-8a3d-8b59c21da820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:24.010819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb5671-3d1c-4b5d-932e-033a2e6943ea service nova] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:24.020537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d8391-71ab-48c4-9417-5c993d5c94d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:24.021591 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d8391-71ab-48c4-9417-5c993d5c94d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:24.023175 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 920/1813] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:24.024512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e82b5-1360-43eb-8a3d-8b59c21da820 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:24.026070 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3d6e82b5-1360-43eb-8a3d-8b59c21da820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:24.032192 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d8391-71ab-48c4-9417-5c993d5c94d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:24.034144 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d8391-71ab-48c4-9417-5c993d5c94d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:24.037174 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65d8391-71ab-48c4-9417-5c993d5c94d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:24.040814 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65d8391-71ab-48c4-9417-5c993d5c94d7 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:24.041318 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e82b5-1360-43eb-8a3d-8b59c21da820 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:24.047389 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15863e81-d341-4a44-9c92-612499f6ba32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:24.048702 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15863e81-d341-4a44-9c92-612499f6ba32 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:24.048986 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15863e81-d341-4a44-9c92-612499f6ba32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:24.049330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15863e81-d341-4a44-9c92-612499f6ba32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:24.050210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15863e81-d341-4a44-9c92-612499f6ba32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:24.051778 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15863e81-d341-4a44-9c92-612499f6ba32 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:24.069337 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 921/1814] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:24.070495 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 894/1815] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:24.071087 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15863e81-d341-4a44-9c92-612499f6ba32 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:24.077182 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d8391-71ab-48c4-9417-5c993d5c94d7 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:24.082332 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-d65d8391-71ab-48c4-9417-5c993d5c94d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:24.090417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15863e81-d341-4a44-9c92-612499f6ba32 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:24.097824 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e24af-a00b-4b48-a146-91877e815c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:24.098578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e24af-a00b-4b48-a146-91877e815c34 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:24.098865 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e24af-a00b-4b48-a146-91877e815c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:24.099143 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e24af-a00b-4b48-a146-91877e815c34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:24.100015 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651e24af-a00b-4b48-a146-91877e815c34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:24.102078 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651e24af-a00b-4b48-a146-91877e815c34 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:24.123466 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 922/1816] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:24.123466 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e24af-a00b-4b48-a146-91877e815c34 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:24.136875 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e24af-a00b-4b48-a146-91877e815c34 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:24.152976 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab8f1ef-1123-4758-91ba-211d38edf0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:24.155031 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8f1ef-1123-4758-91ba-211d38edf0d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:24.155031 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8f1ef-1123-4758-91ba-211d38edf0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:24.155031 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8f1ef-1123-4758-91ba-211d38edf0d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:24.156168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab8f1ef-1123-4758-91ba-211d38edf0d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:24.161566 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab8f1ef-1123-4758-91ba-211d38edf0d7 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:24.179596 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 923/1817] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:24.181340 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab8f1ef-1123-4758-91ba-211d38edf0d7 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:24.195643 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab8f1ef-1123-4758-91ba-211d38edf0d7 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:24.203513 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba618d2-2997-4bdc-a34c-f53e96e01852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:24.204847 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba618d2-2997-4bdc-a34c-f53e96e01852 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:24.205424 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba618d2-2997-4bdc-a34c-f53e96e01852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:24.205703 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba618d2-2997-4bdc-a34c-f53e96e01852 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:24.206529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba618d2-2997-4bdc-a34c-f53e96e01852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:24.209118 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba618d2-2997-4bdc-a34c-f53e96e01852 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:24.227388 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 924/1818] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:24.229675 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba618d2-2997-4bdc-a34c-f53e96e01852 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:24.238414 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d8391-71ab-48c4-9417-5c993d5c94d7 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:24.247506 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee591c14-e3a3-4139-a24c-96ad6fd07405 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:24.248315 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee591c14-e3a3-4139-a24c-96ad6fd07405 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:24.248496 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee591c14-e3a3-4139-a24c-96ad6fd07405 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:24.248770 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee591c14-e3a3-4139-a24c-96ad6fd07405 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:24.252892 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba618d2-2997-4bdc-a34c-f53e96e01852 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:24.260540 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878dcc9e-551b-4e8b-8f06-7805241739b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:24.261982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878dcc9e-551b-4e8b-8f06-7805241739b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:24.262873 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878dcc9e-551b-4e8b-8f06-7805241739b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:24.263348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878dcc9e-551b-4e8b-8f06-7805241739b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:24.264691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878dcc9e-551b-4e8b-8f06-7805241739b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:24.268482 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878dcc9e-551b-4e8b-8f06-7805241739b9 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:24.272829 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee591c14-e3a3-4139-a24c-96ad6fd07405 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:24.276274 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee591c14-e3a3-4139-a24c-96ad6fd07405 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:24.288261 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 925/1819] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:24.292982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878dcc9e-551b-4e8b-8f06-7805241739b9 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:24.295190 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 895/1820] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:24.296526 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee591c14-e3a3-4139-a24c-96ad6fd07405 service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:24.310520 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878dcc9e-551b-4e8b-8f06-7805241739b9 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:24.317679 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f03d1-1c0b-420b-8e2a-727d2a767d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:24.319420 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f03d1-1c0b-420b-8e2a-727d2a767d1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:24.319745 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f03d1-1c0b-420b-8e2a-727d2a767d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:24.320027 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f03d1-1c0b-420b-8e2a-727d2a767d1c admin admin] PATH_INFO: `/v3/projects/88fa24294afe4dce87c5263b482c5679/users/dad0c4050e4348d4aadbd2dbc5d655b9/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:24.341509 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210f03d1-1c0b-420b-8e2a-727d2a767d1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=88fa24294afe4dce87c5263b482c5679, user_id=dad0c4050e4348d4aadbd2dbc5d655b9, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:24.346179 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210f03d1-1c0b-420b-8e2a-727d2a767d1c admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:24.347504 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee591c14-e3a3-4139-a24c-96ad6fd07405 service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:24.370297 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 926/1821] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:24 2022] PUT /identity/v3/projects/88fa24294afe4dce87c5263b482c5679/users/dad0c4050e4348d4aadbd2dbc5d655b9/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:24.380520 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369effa7-9e38-489f-b7e0-9d9c6df80dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:24.381997 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369effa7-9e38-489f-b7e0-9d9c6df80dd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:24.382199 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f03d1-1c0b-420b-8e2a-727d2a767d1c admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:24.382444 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369effa7-9e38-489f-b7e0-9d9c6df80dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:24.383051 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369effa7-9e38-489f-b7e0-9d9c6df80dd2 admin admin] PATH_INFO: `/v3/projects/aa1b82105e884be5a2629c53dbf96fa5/users/62ee3f143a9e4bfa852c7e794dd7d990/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:24.437083 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369effa7-9e38-489f-b7e0-9d9c6df80dd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa1b82105e884be5a2629c53dbf96fa5, user_id=62ee3f143a9e4bfa852c7e794dd7d990, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:24.443894 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369effa7-9e38-489f-b7e0-9d9c6df80dd2 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:24.483457 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 896/1822] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:24 2022] PUT /identity/v3/projects/aa1b82105e884be5a2629c53dbf96fa5/users/62ee3f143a9e4bfa852c7e794dd7d990/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:24.487766 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67f4de-46fd-4f92-9128-274307670dbe None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:24.487766 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67f4de-46fd-4f92-9128-274307670dbe None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:24.487766 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67f4de-46fd-4f92-9128-274307670dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:24.504046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:24.522756 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f03d1-1c0b-420b-8e2a-727d2a767d1c admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:24.550933 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-6a67f4de-46fd-4f92-9128-274307670dbe None None] MFA Rules not processed for user `dad0c4050e4348d4aadbd2dbc5d655b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:24.553701 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-6a67f4de-46fd-4f92-9128-274307670dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:24.592192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52365ce3-8024-4ed8-8de2-ae629663bcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:24.592890 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52365ce3-8024-4ed8-8de2-ae629663bcf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:24.593172 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52365ce3-8024-4ed8-8de2-ae629663bcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:24.593453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52365ce3-8024-4ed8-8de2-ae629663bcf3 admin admin] PATH_INFO: `/v3/projects/76d08d06cd7342f0a356160b2c371234/users/7b07b31e3d384a0e81039a7e99d405a0/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:24.602113 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:24.612021 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52365ce3-8024-4ed8-8de2-ae629663bcf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d08d06cd7342f0a356160b2c371234, user_id=7b07b31e3d384a0e81039a7e99d405a0, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:24.614997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52365ce3-8024-4ed8-8de2-ae629663bcf3 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:24.640898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 927/1823] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:24 2022] PUT /identity/v3/projects/76d08d06cd7342f0a356160b2c371234/users/7b07b31e3d384a0e81039a7e99d405a0/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:24.643960 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a258b7-9a58-417c-a5aa-9b415153b0e4 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:24.645988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a258b7-9a58-417c-a5aa-9b415153b0e4 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:24.646503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a258b7-9a58-417c-a5aa-9b415153b0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:24.690857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 897/1824] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:24.690857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf6745-6712-4050-b8f0-2d3d09dc31ac None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:24.690857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf6745-6712-4050-b8f0-2d3d09dc31ac None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:24.690857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf6745-6712-4050-b8f0-2d3d09dc31ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:24.728149 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-97cf6745-6712-4050-b8f0-2d3d09dc31ac None None] MFA Rules not processed for user `7b07b31e3d384a0e81039a7e99d405a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:24.729685 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-97cf6745-6712-4050-b8f0-2d3d09dc31ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:24.730796 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-51a258b7-9a58-417c-a5aa-9b415153b0e4 None None] MFA Rules not processed for user `62ee3f143a9e4bfa852c7e794dd7d990`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:24.736953 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-51a258b7-9a58-417c-a5aa-9b415153b0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:24.888936 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 898/1825] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:24.890132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf6745-6712-4050-b8f0-2d3d09dc31ac None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:24.934022 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 928/1826] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:24 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:24.935829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a258b7-9a58-417c-a5aa-9b415153b0e4 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:25.029876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf6745-6712-4050-b8f0-2d3d09dc31ac None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:25.035627 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c853406c-c6a2-45ad-be57-8551d34e22bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:25.036266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c853406c-c6a2-45ad-be57-8551d34e22bc admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:25.037117 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c853406c-c6a2-45ad-be57-8551d34e22bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:25.037117 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c853406c-c6a2-45ad-be57-8551d34e22bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:25.037856 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c853406c-c6a2-45ad-be57-8551d34e22bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:25.040219 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c853406c-c6a2-45ad-be57-8551d34e22bc admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:25.083871 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 899/1827] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:24 2022] POST /identity/v3/projects => generated 357 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:25.087928 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c853406c-c6a2-45ad-be57-8551d34e22bc admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:25.103685 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a258b7-9a58-417c-a5aa-9b415153b0e4 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:25.135005 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:25.151649 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:25.185280 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c853406c-c6a2-45ad-be57-8551d34e22bc admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:25.192276 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5976c-58f7-4d21-9a95-5afbf0b36b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:25.193483 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5976c-58f7-4d21-9a95-5afbf0b36b24 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:25.193986 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5976c-58f7-4d21-9a95-5afbf0b36b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:25.194846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5976c-58f7-4d21-9a95-5afbf0b36b24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:25.195923 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a5976c-58f7-4d21-9a95-5afbf0b36b24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:25.198500 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a5976c-58f7-4d21-9a95-5afbf0b36b24 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:25.220463 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed4933-d8eb-4011-adf6-4307a186121f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:25.221763 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed4933-d8eb-4011-adf6-4307a186121f admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:25.222274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed4933-d8eb-4011-adf6-4307a186121f admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:25.222746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed4933-d8eb-4011-adf6-4307a186121f admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:25.224038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ed4933-d8eb-4011-adf6-4307a186121f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:25.227916 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ed4933-d8eb-4011-adf6-4307a186121f admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:25.240731 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 900/1828] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:25 2022] POST /identity/v3/projects => generated 399 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:25.242205 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5976c-58f7-4d21-9a95-5afbf0b36b24 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:25.281720 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 929/1829] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:24 2022] POST /identity/v3/projects => generated 367 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:25.282644 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed4933-d8eb-4011-adf6-4307a186121f admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:25.396544 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed4933-d8eb-4011-adf6-4307a186121f admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:25.403738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5976c-58f7-4d21-9a95-5afbf0b36b24 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:25.403990 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497eece4-8cab-4141-a048-f92635dbacb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:25.404603 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497eece4-8cab-4141-a048-f92635dbacb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:25.404886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497eece4-8cab-4141-a048-f92635dbacb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:25.405254 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497eece4-8cab-4141-a048-f92635dbacb2 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:25.405812 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497eece4-8cab-4141-a048-f92635dbacb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:25.408087 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497eece4-8cab-4141-a048-f92635dbacb2 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:25.414704 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12ce39-508a-4966-99e3-0035610612cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:25.415517 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12ce39-508a-4966-99e3-0035610612cf admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:25.415906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12ce39-508a-4966-99e3-0035610612cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:25.416311 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12ce39-508a-4966-99e3-0035610612cf admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:25.417321 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe12ce39-508a-4966-99e3-0035610612cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:25.420096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe12ce39-508a-4966-99e3-0035610612cf admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:25.440380 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 930/1830] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:25 2022] POST /identity/v3/users => generated 306 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:25.441213 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497eece4-8cab-4141-a048-f92635dbacb2 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:25.458624 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497eece4-8cab-4141-a048-f92635dbacb2 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:25.466561 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 901/1831] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:25 2022] POST /identity/v3/users => generated 285 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:25.467326 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12ce39-508a-4966-99e3-0035610612cf admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:25.469865 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f758e7-f6d1-44ff-a9de-f3132d516f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:25.471132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f758e7-f6d1-44ff-a9de-f3132d516f52 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:25.471995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f758e7-f6d1-44ff-a9de-f3132d516f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:25.472368 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f758e7-f6d1-44ff-a9de-f3132d516f52 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:25.473653 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f758e7-f6d1-44ff-a9de-f3132d516f52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:25.474685 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-fe12ce39-508a-4966-99e3-0035610612cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:25.477769 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f758e7-f6d1-44ff-a9de-f3132d516f52 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:25.492937 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12ce39-508a-4966-99e3-0035610612cf admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:25.499982 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2439ec6-ae57-49e2-b41e-867825f03361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:25.500799 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2439ec6-ae57-49e2-b41e-867825f03361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:25.501735 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2439ec6-ae57-49e2-b41e-867825f03361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:25.502083 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2439ec6-ae57-49e2-b41e-867825f03361 admin admin] PATH_INFO: `/v3/projects/b30af7a2c7264080a2dc2d80c17b07a9` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:25.503794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2439ec6-ae57-49e2-b41e-867825f03361 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b30af7a2c7264080a2dc2d80c17b07a9)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:25.506037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2439ec6-ae57-49e2-b41e-867825f03361 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:25.524463 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 931/1832] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:25 2022] POST /identity/v3/users => generated 290 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:25.527554 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f758e7-f6d1-44ff-a9de-f3132d516f52 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:25.550147 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f758e7-f6d1-44ff-a9de-f3132d516f52 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:25.555896 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c132c6-d45a-4148-b50d-f838cbc704ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:25.556718 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c132c6-d45a-4148-b50d-f838cbc704ec admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:25.557128 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c132c6-d45a-4148-b50d-f838cbc704ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:25.557613 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c132c6-d45a-4148-b50d-f838cbc704ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:25.558496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c132c6-d45a-4148-b50d-f838cbc704ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:25.560809 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c132c6-d45a-4148-b50d-f838cbc704ec admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:25.581402 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 932/1833] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:25.583120 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c132c6-d45a-4148-b50d-f838cbc704ec admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:25.584843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b2439ec6-ae57-49e2-b41e-867825f03361 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b30af7a2c7264080a2dc2d80c17b07a9', 'request_id': 'req-b2439ec6-ae57-49e2-b41e-867825f03361'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:25.598789 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c132c6-d45a-4148-b50d-f838cbc704ec admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:25.606497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b2439ec6-ae57-49e2-b41e-867825f03361 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b30af7a2c7264080a2dc2d80c17b07a9', 'request_id': 'req-b2439ec6-ae57-49e2-b41e-867825f03361'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:25.618937 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b2439ec6-ae57-49e2-b41e-867825f03361 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b30af7a2c7264080a2dc2d80c17b07a9', 'request_id': 'req-b2439ec6-ae57-49e2-b41e-867825f03361'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:25.643454 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b2439ec6-ae57-49e2-b41e-867825f03361 admin admin] The token cache is being invalidate because project b30af7a2c7264080a2dc2d80c17b07a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:25.644070 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b2439ec6-ae57-49e2-b41e-867825f03361 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:25.645108 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b2439ec6-ae57-49e2-b41e-867825f03361 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:25.647869 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 902/1834] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:25 2022] DELETE /identity/v3/projects/b30af7a2c7264080a2dc2d80c17b07a9 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:25.652533 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2439ec6-ae57-49e2-b41e-867825f03361 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:25.655171 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-b2439ec6-ae57-49e2-b41e-867825f03361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:25.740929 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:25.744815 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40eb97c-5825-4010-815a-d5ba9f06b8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:25.747291 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40eb97c-5825-4010-815a-d5ba9f06b8a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:25.747571 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40eb97c-5825-4010-815a-d5ba9f06b8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:25.747972 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40eb97c-5825-4010-815a-d5ba9f06b8a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:25.749199 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40eb97c-5825-4010-815a-d5ba9f06b8a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:25.752644 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40eb97c-5825-4010-815a-d5ba9f06b8a1 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:25.766436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 933/1835] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:25.767891 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40eb97c-5825-4010-815a-d5ba9f06b8a1 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:25.769982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e40eb97c-5825-4010-815a-d5ba9f06b8a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:25.778375 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:25.788749 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40eb97c-5825-4010-815a-d5ba9f06b8a1 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:25.796776 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64157c19-d77c-4c85-873e-0d34916615eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:25.797804 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64157c19-d77c-4c85-873e-0d34916615eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:25.798809 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64157c19-d77c-4c85-873e-0d34916615eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:25.799238 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64157c19-d77c-4c85-873e-0d34916615eb admin admin] PATH_INFO: `/v3/projects/79626f372a5044df8f88e3fd37a98d98/users/69a83cba7cf94579998f9e24edbd2adc/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:25.801947 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2439ec6-ae57-49e2-b41e-867825f03361 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:25.810089 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03294e-4613-4f04-a417-89e63d4e3a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:25.811068 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03294e-4613-4f04-a417-89e63d4e3a56 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:25.811547 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03294e-4613-4f04-a417-89e63d4e3a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:25.812005 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03294e-4613-4f04-a417-89e63d4e3a56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:25.813119 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b03294e-4613-4f04-a417-89e63d4e3a56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:25.816627 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b03294e-4613-4f04-a417-89e63d4e3a56 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:25.823972 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64157c19-d77c-4c85-873e-0d34916615eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=79626f372a5044df8f88e3fd37a98d98, user_id=69a83cba7cf94579998f9e24edbd2adc, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:25.827172 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64157c19-d77c-4c85-873e-0d34916615eb admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:25.830450 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 903/1836] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:25.831231 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03294e-4613-4f04-a417-89e63d4e3a56 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:25.833095 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-6b03294e-4613-4f04-a417-89e63d4e3a56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:25.850602 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 934/1837] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:25 2022] PUT /identity/v3/projects/79626f372a5044df8f88e3fd37a98d98/users/69a83cba7cf94579998f9e24edbd2adc/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:25.851285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03294e-4613-4f04-a417-89e63d4e3a56 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:25.851396 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64157c19-d77c-4c85-873e-0d34916615eb admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:25.853425 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-64157c19-d77c-4c85-873e-0d34916615eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:25.922531 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3499c98-d115-454d-b0a7-b7d13fc0c5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:25.923738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3499c98-d115-454d-b0a7-b7d13fc0c5b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:25.924084 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3499c98-d115-454d-b0a7-b7d13fc0c5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:25.924392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3499c98-d115-454d-b0a7-b7d13fc0c5b1 admin admin] PATH_INFO: `/v3/users/12cd5402bd0442b0b2e71e7f933f3518` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:25.926429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3499c98-d115-454d-b0a7-b7d13fc0c5b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12cd5402bd0442b0b2e71e7f933f3518)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:25.929760 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3499c98-d115-454d-b0a7-b7d13fc0c5b1 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:25.964970 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64157c19-d77c-4c85-873e-0d34916615eb admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:25.971334 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9004c1b-3ab3-4898-95d2-83ffea2cb8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:25.971897 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9004c1b-3ab3-4898-95d2-83ffea2cb8a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:25.972076 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9004c1b-3ab3-4898-95d2-83ffea2cb8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:25.972482 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9004c1b-3ab3-4898-95d2-83ffea2cb8a5 admin admin] PATH_INFO: `/v3/projects/e776245c799148588827e227c754d0d7/users/5cf0980412b54ba1960017841ab87b8f/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:25.992214 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9004c1b-3ab3-4898-95d2-83ffea2cb8a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e776245c799148588827e227c754d0d7, user_id=5cf0980412b54ba1960017841ab87b8f, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:25.994985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9004c1b-3ab3-4898-95d2-83ffea2cb8a5 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:26.012473 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 935/1838] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:25 2022] PUT /identity/v3/projects/e776245c799148588827e227c754d0d7/users/5cf0980412b54ba1960017841ab87b8f/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:26.013316 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9004c1b-3ab3-4898-95d2-83ffea2cb8a5 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:26.019329 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-f3499c98-d115-454d-b0a7-b7d13fc0c5b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12cd5402bd0442b0b2e71e7f933f3518'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:26.033288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-f3499c98-d115-454d-b0a7-b7d13fc0c5b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12cd5402bd0442b0b2e71e7f933f3518'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:26.034060 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-f3499c98-d115-454d-b0a7-b7d13fc0c5b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12cd5402bd0442b0b2e71e7f933f3518'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:26.063394 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-f3499c98-d115-454d-b0a7-b7d13fc0c5b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12cd5402bd0442b0b2e71e7f933f3518'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:26.079834 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-f3499c98-d115-454d-b0a7-b7d13fc0c5b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12cd5402bd0442b0b2e71e7f933f3518'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:26.093528 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 904/1839] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:25 2022] DELETE /identity/v3/users/12cd5402bd0442b0b2e71e7f933f3518 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:26.095085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3499c98-d115-454d-b0a7-b7d13fc0c5b1 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:26.098640 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-f3499c98-d115-454d-b0a7-b7d13fc0c5b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:26.122619 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9004c1b-3ab3-4898-95d2-83ffea2cb8a5 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:26.183209 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:26.202330 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3499c98-d115-454d-b0a7-b7d13fc0c5b1 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:26.209329 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c651184-26a0-4b2c-a7bc-2ef431c35848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:26.210143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c651184-26a0-4b2c-a7bc-2ef431c35848 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:26.210529 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c651184-26a0-4b2c-a7bc-2ef431c35848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:26.210916 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c651184-26a0-4b2c-a7bc-2ef431c35848 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:26.211994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c651184-26a0-4b2c-a7bc-2ef431c35848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:26.214198 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c651184-26a0-4b2c-a7bc-2ef431c35848 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:26.228048 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 905/1840] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:26.232562 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c651184-26a0-4b2c-a7bc-2ef431c35848 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:26.234619 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-4c651184-26a0-4b2c-a7bc-2ef431c35848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:26.254605 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff04a49-2554-44d3-9638-df6fd4ce5451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:26.255354 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff04a49-2554-44d3-9638-df6fd4ce5451 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:26.255667 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff04a49-2554-44d3-9638-df6fd4ce5451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:26.255941 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff04a49-2554-44d3-9638-df6fd4ce5451 admin admin] PATH_INFO: `/v3/projects/b96b2c84f81d47afb301304a4ab486a0/users/0a73a00c3ce4496ebda509a7dde95b39/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:26.256743 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c651184-26a0-4b2c-a7bc-2ef431c35848 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:26.264325 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5820d0-ec04-43ad-84cb-b91137c1cb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:26.265066 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5820d0-ec04-43ad-84cb-b91137c1cb0c admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:26.265441 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5820d0-ec04-43ad-84cb-b91137c1cb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:26.266077 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5820d0-ec04-43ad-84cb-b91137c1cb0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:26.266880 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5820d0-ec04-43ad-84cb-b91137c1cb0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:26.268716 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5820d0-ec04-43ad-84cb-b91137c1cb0c admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:26.275227 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:26.279211 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff04a49-2554-44d3-9638-df6fd4ce5451 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b96b2c84f81d47afb301304a4ab486a0, user_id=0a73a00c3ce4496ebda509a7dde95b39, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:26.282133 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff04a49-2554-44d3-9638-df6fd4ce5451 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:26.283622 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 906/1841] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:26.284907 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5820d0-ec04-43ad-84cb-b91137c1cb0c admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:26.290267 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-5d5820d0-ec04-43ad-84cb-b91137c1cb0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:26.303603 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 936/1842] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:26 2022] PUT /identity/v3/projects/b96b2c84f81d47afb301304a4ab486a0/users/0a73a00c3ce4496ebda509a7dde95b39/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:26.304827 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff04a49-2554-44d3-9638-df6fd4ce5451 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:26.311064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5820d0-ec04-43ad-84cb-b91137c1cb0c admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:26.391549 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff04a49-2554-44d3-9638-df6fd4ce5451 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:26.398649 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e91b1-6e55-4d9e-9924-2a56adc193c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:26.399975 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e91b1-6e55-4d9e-9924-2a56adc193c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:26.400138 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e91b1-6e55-4d9e-9924-2a56adc193c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:26.400310 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e91b1-6e55-4d9e-9924-2a56adc193c3 admin admin] PATH_INFO: `/v3/projects/79626f372a5044df8f88e3fd37a98d98/users/69a83cba7cf94579998f9e24edbd2adc/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:26.402712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e91b1-6e55-4d9e-9924-2a56adc193c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79626f372a5044df8f88e3fd37a98d98, user_id=69a83cba7cf94579998f9e24edbd2adc, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:26.405724 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e91b1-6e55-4d9e-9924-2a56adc193c3 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:26.419680 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 937/1843] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:26 2022] PUT /identity/v3/projects/79626f372a5044df8f88e3fd37a98d98/users/69a83cba7cf94579998f9e24edbd2adc/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:26.429152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e91b1-6e55-4d9e-9924-2a56adc193c3 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:26.446907 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e05da-7b14-4f65-8ea4-28365014472b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:26.448379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e05da-7b14-4f65-8ea4-28365014472b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:26.449313 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e05da-7b14-4f65-8ea4-28365014472b admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:26.449999 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e05da-7b14-4f65-8ea4-28365014472b admin admin] PATH_INFO: `/v3/users/bfacaab427ac4562bae47fe50ea909c3` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:26.452444 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986e05da-7b14-4f65-8ea4-28365014472b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfacaab427ac4562bae47fe50ea909c3)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:26.455737 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986e05da-7b14-4f65-8ea4-28365014472b admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:26.510377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e91b1-6e55-4d9e-9924-2a56adc193c3 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:26.518777 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de26b3-45c4-482d-aec5-2f7a8888124e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:26.519697 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de26b3-45c4-482d-aec5-2f7a8888124e admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:26.520155 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de26b3-45c4-482d-aec5-2f7a8888124e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:26.523428 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de26b3-45c4-482d-aec5-2f7a8888124e admin admin] PATH_INFO: `/v3/projects/e776245c799148588827e227c754d0d7/users/5cf0980412b54ba1960017841ab87b8f/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:26.525229 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de26b3-45c4-482d-aec5-2f7a8888124e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e776245c799148588827e227c754d0d7, user_id=5cf0980412b54ba1960017841ab87b8f, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:26.528865 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de26b3-45c4-482d-aec5-2f7a8888124e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:26.545855 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 938/1844] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:26 2022] PUT /identity/v3/projects/e776245c799148588827e227c754d0d7/users/5cf0980412b54ba1960017841ab87b8f/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:26.547026 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de26b3-45c4-482d-aec5-2f7a8888124e admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:26.549740 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-19de26b3-45c4-482d-aec5-2f7a8888124e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:26.569490 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-986e05da-7b14-4f65-8ea4-28365014472b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfacaab427ac4562bae47fe50ea909c3'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:26.580884 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-986e05da-7b14-4f65-8ea4-28365014472b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfacaab427ac4562bae47fe50ea909c3'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:26.581726 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-986e05da-7b14-4f65-8ea4-28365014472b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfacaab427ac4562bae47fe50ea909c3'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:26.612989 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-986e05da-7b14-4f65-8ea4-28365014472b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfacaab427ac4562bae47fe50ea909c3'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:26.637297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-986e05da-7b14-4f65-8ea4-28365014472b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfacaab427ac4562bae47fe50ea909c3'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:26.653522 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 907/1845] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:26 2022] DELETE /identity/v3/users/bfacaab427ac4562bae47fe50ea909c3 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:26.655000 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ab58d-b1c3-4b00-9e49-ea4140a46297 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:26.655691 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ab58d-b1c3-4b00-9e49-ea4140a46297 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:26.656199 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ab58d-b1c3-4b00-9e49-ea4140a46297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:26.663610 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de26b3-45c4-482d-aec5-2f7a8888124e admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:26.689211 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-d81ab58d-b1c3-4b00-9e49-ea4140a46297 None None] MFA Rules not processed for user `69a83cba7cf94579998f9e24edbd2adc`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:26.691698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-d81ab58d-b1c3-4b00-9e49-ea4140a46297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:26.696905 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:26.747725 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:26.749769 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b36c5-0796-4266-9dee-e86a22cd7887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:26.751247 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b36c5-0796-4266-9dee-e86a22cd7887 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:26.751444 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b36c5-0796-4266-9dee-e86a22cd7887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:26.752015 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b36c5-0796-4266-9dee-e86a22cd7887 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:26.753389 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b36c5-0796-4266-9dee-e86a22cd7887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:26.756671 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b36c5-0796-4266-9dee-e86a22cd7887 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:26.778846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 908/1846] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:26 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:26.783278 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c07bb-84e0-4fde-982b-89859098765f None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:26.783278 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c07bb-84e0-4fde-982b-89859098765f None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:26.783278 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c07bb-84e0-4fde-982b-89859098765f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:26.790320 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 939/1847] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:26.791901 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b36c5-0796-4266-9dee-e86a22cd7887 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:26.796146 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-418b36c5-0796-4266-9dee-e86a22cd7887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:26.822425 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-086c07bb-84e0-4fde-982b-89859098765f None None] MFA Rules not processed for user `5cf0980412b54ba1960017841ab87b8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:26.823706 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b36c5-0796-4266-9dee-e86a22cd7887 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:26.824046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-086c07bb-84e0-4fde-982b-89859098765f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:26.830522 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f4840-7906-46a7-bc29-01400d042b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:26.832200 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f4840-7906-46a7-bc29-01400d042b9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:26.833012 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f4840-7906-46a7-bc29-01400d042b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:26.833511 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f4840-7906-46a7-bc29-01400d042b9f admin admin] PATH_INFO: `/v3/projects/0fab995ba2e04d19bf15930702a40d96` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:26.835410 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f4840-7906-46a7-bc29-01400d042b9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fab995ba2e04d19bf15930702a40d96)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:26.837692 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f4840-7906-46a7-bc29-01400d042b9f admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:26.924867 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-841f4840-7906-46a7-bc29-01400d042b9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fab995ba2e04d19bf15930702a40d96', 'request_id': 'req-841f4840-7906-46a7-bc29-01400d042b9f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:26.943619 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-841f4840-7906-46a7-bc29-01400d042b9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fab995ba2e04d19bf15930702a40d96', 'request_id': 'req-841f4840-7906-46a7-bc29-01400d042b9f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:26.945134 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-841f4840-7906-46a7-bc29-01400d042b9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fab995ba2e04d19bf15930702a40d96', 'request_id': 'req-841f4840-7906-46a7-bc29-01400d042b9f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:26.960126 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 909/1848] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:26 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:26.962117 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb355c-8713-4d26-81e4-481936d3376f None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:26.962117 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb355c-8713-4d26-81e4-481936d3376f None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:26.962475 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb355c-8713-4d26-81e4-481936d3376f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:26.977953 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-841f4840-7906-46a7-bc29-01400d042b9f admin admin] The token cache is being invalidate because project 0fab995ba2e04d19bf15930702a40d96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:26.977953 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-841f4840-7906-46a7-bc29-01400d042b9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:26.978210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-841f4840-7906-46a7-bc29-01400d042b9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:26.983633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 940/1849] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:26 2022] DELETE /identity/v3/projects/0fab995ba2e04d19bf15930702a40d96 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:26.983633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f4840-7906-46a7-bc29-01400d042b9f admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:26.983633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-841f4840-7906-46a7-bc29-01400d042b9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:26.985912 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-e1eb355c-8713-4d26-81e4-481936d3376f None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:26.987196 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-e1eb355c-8713-4d26-81e4-481936d3376f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:27.060013 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f4840-7906-46a7-bc29-01400d042b9f admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:27.068891 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9cbbb-f2a4-4394-9af4-89d8f0c37283 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:27.069998 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9cbbb-f2a4-4394-9af4-89d8f0c37283 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:27.070480 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9cbbb-f2a4-4394-9af4-89d8f0c37283 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:27.070980 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9cbbb-f2a4-4394-9af4-89d8f0c37283 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:27.075391 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 910/1850] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:27.075391 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb355c-8713-4d26-81e4-481936d3376f None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:27.077815 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-dca9cbbb-f2a4-4394-9af4-89d8f0c37283 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:27.080786 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-e1eb355c-8713-4d26-81e4-481936d3376f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:27.096260 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb355c-8713-4d26-81e4-481936d3376f None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:27.102394 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a806a-d926-4f22-8dbb-daaa422962da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:27.103276 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a806a-d926-4f22-8dbb-daaa422962da admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:27.103671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a806a-d926-4f22-8dbb-daaa422962da admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:27.103866 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a806a-d926-4f22-8dbb-daaa422962da admin admin] PATH_INFO: `/v3/projects/b96b2c84f81d47afb301304a4ab486a0/users/0a73a00c3ce4496ebda509a7dde95b39/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:27.106142 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6a806a-d926-4f22-8dbb-daaa422962da admin admin] RBAC: Authorizing `identity:create_grant(project_id=b96b2c84f81d47afb301304a4ab486a0, user_id=0a73a00c3ce4496ebda509a7dde95b39, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:27.109083 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6a806a-d926-4f22-8dbb-daaa422962da admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:27.123765 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 911/1851] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:27 2022] PUT /identity/v3/projects/b96b2c84f81d47afb301304a4ab486a0/users/0a73a00c3ce4496ebda509a7dde95b39/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:27.125831 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a806a-d926-4f22-8dbb-daaa422962da admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:27.129152 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-7a6a806a-d926-4f22-8dbb-daaa422962da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:27.187022 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:27.192191 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca9cbbb-f2a4-4394-9af4-89d8f0c37283 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:27.195677 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca9cbbb-f2a4-4394-9af4-89d8f0c37283 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:27.210118 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a806a-d926-4f22-8dbb-daaa422962da admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:27.216232 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 941/1852] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:26 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:27.216916 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c6ffa-f268-4b5c-9855-6b0b98a9da61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:27.217694 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9cbbb-f2a4-4394-9af4-89d8f0c37283 service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:27.217928 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c6ffa-f268-4b5c-9855-6b0b98a9da61 service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:27.218427 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c6ffa-f268-4b5c-9855-6b0b98a9da61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:27.218872 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c6ffa-f268-4b5c-9855-6b0b98a9da61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:27.220685 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-dca9cbbb-f2a4-4394-9af4-89d8f0c37283 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:27.223076 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-0a8c6ffa-f268-4b5c-9855-6b0b98a9da61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:27.234777 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:27.248799 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9cbbb-f2a4-4394-9af4-89d8f0c37283 service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:27.322809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c6ffa-f268-4b5c-9855-6b0b98a9da61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:27.324313 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c6ffa-f268-4b5c-9855-6b0b98a9da61 service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:27.335631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 912/1853] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:27 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:27.336422 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c6ffa-f268-4b5c-9855-6b0b98a9da61 service cinder] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:27.337822 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-0a8c6ffa-f268-4b5c-9855-6b0b98a9da61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:27.358390 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc91a5-223e-4724-a1f9-da4159fd688e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:27.359046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc91a5-223e-4724-a1f9-da4159fd688e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:27.360064 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc91a5-223e-4724-a1f9-da4159fd688e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:27.360064 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc91a5-223e-4724-a1f9-da4159fd688e admin admin] PATH_INFO: `/v3/users/a03e38fb0fbd40b6a2da1669774f9bb6` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:27.362179 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc91a5-223e-4724-a1f9-da4159fd688e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a03e38fb0fbd40b6a2da1669774f9bb6)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:27.365281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc91a5-223e-4724-a1f9-da4159fd688e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:27.434604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c6ffa-f268-4b5c-9855-6b0b98a9da61 service cinder] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:27.442020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ecded3-037e-4176-8119-9ac0bb43a04b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:27.443148 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ecded3-037e-4176-8119-9ac0bb43a04b service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:27.443625 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ecded3-037e-4176-8119-9ac0bb43a04b service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:27.444077 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ecded3-037e-4176-8119-9ac0bb43a04b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:27.460707 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ecded3-037e-4176-8119-9ac0bb43a04b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:27.461823 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fdcc91a5-223e-4724-a1f9-da4159fd688e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a03e38fb0fbd40b6a2da1669774f9bb6'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:27.462667 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ecded3-037e-4176-8119-9ac0bb43a04b service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:27.473632 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fdcc91a5-223e-4724-a1f9-da4159fd688e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a03e38fb0fbd40b6a2da1669774f9bb6'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:27.474628 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fdcc91a5-223e-4724-a1f9-da4159fd688e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a03e38fb0fbd40b6a2da1669774f9bb6'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:27.475645 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fdcc91a5-223e-4724-a1f9-da4159fd688e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a03e38fb0fbd40b6a2da1669774f9bb6'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:27.477491 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 913/1854] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:27.488203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049abc0-c3b7-44db-80a3-8aa65878fadb None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:27.489023 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049abc0-c3b7-44db-80a3-8aa65878fadb None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:27.490437 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049abc0-c3b7-44db-80a3-8aa65878fadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:27.506088 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fdcc91a5-223e-4724-a1f9-da4159fd688e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a03e38fb0fbd40b6a2da1669774f9bb6'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:27.541484 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 942/1855] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:27 2022] DELETE /identity/v3/users/a03e38fb0fbd40b6a2da1669774f9bb6 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:27.544998 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc91a5-223e-4724-a1f9-da4159fd688e admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:27.548703 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-fdcc91a5-223e-4724-a1f9-da4159fd688e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:27.563373 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-a049abc0-c3b7-44db-80a3-8aa65878fadb None None] MFA Rules not processed for user `0a73a00c3ce4496ebda509a7dde95b39`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:27.567001 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-a049abc0-c3b7-44db-80a3-8aa65878fadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:27.645244 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc91a5-223e-4724-a1f9-da4159fd688e admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:27.651983 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee5694-7bba-4783-bdf2-f9c918a7352e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:27.653448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee5694-7bba-4783-bdf2-f9c918a7352e service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:27.653448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee5694-7bba-4783-bdf2-f9c918a7352e service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:27.653448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee5694-7bba-4783-bdf2-f9c918a7352e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:27.655129 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-aeee5694-7bba-4783-bdf2-f9c918a7352e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:27.702754 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:27.762540 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 914/1856] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:27 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:27.766566 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049abc0-c3b7-44db-80a3-8aa65878fadb None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:27.770575 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-a049abc0-c3b7-44db-80a3-8aa65878fadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:27.787122 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee5694-7bba-4783-bdf2-f9c918a7352e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:27.794047 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee5694-7bba-4783-bdf2-f9c918a7352e service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:27.804553 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049abc0-c3b7-44db-80a3-8aa65878fadb None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:27.816795 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faedfb53-7468-48b3-9b4d-5678fa4f76a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:27.818195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 943/1857] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:27 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:27.820288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faedfb53-7468-48b3-9b4d-5678fa4f76a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:27.822162 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faedfb53-7468-48b3-9b4d-5678fa4f76a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:27.823042 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faedfb53-7468-48b3-9b4d-5678fa4f76a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:27.826555 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-faedfb53-7468-48b3-9b4d-5678fa4f76a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:27.911427 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:27.972583 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faedfb53-7468-48b3-9b4d-5678fa4f76a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:27.974247 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faedfb53-7468-48b3-9b4d-5678fa4f76a7 service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:28.004712 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 915/1858] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:27 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:28.313391 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee5694-7bba-4783-bdf2-f9c918a7352e service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:28.325364 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee5694-7bba-4783-bdf2-f9c918a7352e service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:28.330791 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570a34d-1379-4cab-b85f-45fbb79d200b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:28.331606 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570a34d-1379-4cab-b85f-45fbb79d200b service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:28.332036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570a34d-1379-4cab-b85f-45fbb79d200b service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:28.332470 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570a34d-1379-4cab-b85f-45fbb79d200b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:28.347820 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c570a34d-1379-4cab-b85f-45fbb79d200b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:28.348831 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c570a34d-1379-4cab-b85f-45fbb79d200b service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:28.387309 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 944/1859] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:28 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:28.466863 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faedfb53-7468-48b3-9b4d-5678fa4f76a7 service cinder] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:28.477898 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faedfb53-7468-48b3-9b4d-5678fa4f76a7 service cinder] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:28.484145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570a34d-1379-4cab-b85f-45fbb79d200b service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:28.486616 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c570a34d-1379-4cab-b85f-45fbb79d200b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:28.498473 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c5fcf-048f-4761-8f1b-8fe81c1ef493 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:28.500774 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c5fcf-048f-4761-8f1b-8fe81c1ef493 service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:28.500972 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c5fcf-048f-4761-8f1b-8fe81c1ef493 service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:28.501737 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c5fcf-048f-4761-8f1b-8fe81c1ef493 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:28.508851 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-a35c5fcf-048f-4761-8f1b-8fe81c1ef493 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:28.677315 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35c5fcf-048f-4761-8f1b-8fe81c1ef493 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:28.679016 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35c5fcf-048f-4761-8f1b-8fe81c1ef493 service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:28.681544 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570a34d-1379-4cab-b85f-45fbb79d200b service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:28.689915 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f74c0-cd96-406e-a521-d2a88a9f72e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:28.691870 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f74c0-cd96-406e-a521-d2a88a9f72e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:28.692135 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f74c0-cd96-406e-a521-d2a88a9f72e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:28.692393 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 916/1860] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:28.692560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f74c0-cd96-406e-a521-d2a88a9f72e3 admin admin] PATH_INFO: `/v3/projects/48b1cbb8b8504c1da4fc4a54ede3a549` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:28.693742 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29f74c0-cd96-406e-a521-d2a88a9f72e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48b1cbb8b8504c1da4fc4a54ede3a549)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:28.695987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29f74c0-cd96-406e-a521-d2a88a9f72e3 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:28.776053 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b29f74c0-cd96-406e-a521-d2a88a9f72e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48b1cbb8b8504c1da4fc4a54ede3a549', 'request_id': 'req-b29f74c0-cd96-406e-a521-d2a88a9f72e3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:28.802466 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b29f74c0-cd96-406e-a521-d2a88a9f72e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48b1cbb8b8504c1da4fc4a54ede3a549', 'request_id': 'req-b29f74c0-cd96-406e-a521-d2a88a9f72e3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:28.811815 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c5fcf-048f-4761-8f1b-8fe81c1ef493 service cinder] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:28.814368 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-a35c5fcf-048f-4761-8f1b-8fe81c1ef493 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:28.815239 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b29f74c0-cd96-406e-a521-d2a88a9f72e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48b1cbb8b8504c1da4fc4a54ede3a549', 'request_id': 'req-b29f74c0-cd96-406e-a521-d2a88a9f72e3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:28.833565 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b29f74c0-cd96-406e-a521-d2a88a9f72e3 admin admin] The token cache is being invalidate because project 48b1cbb8b8504c1da4fc4a54ede3a549 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:28.834070 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b29f74c0-cd96-406e-a521-d2a88a9f72e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:28.834764 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b29f74c0-cd96-406e-a521-d2a88a9f72e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:28.836766 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 945/1861] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:28 2022] DELETE /identity/v3/projects/48b1cbb8b8504c1da4fc4a54ede3a549 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:19:28.842970 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f74c0-cd96-406e-a521-d2a88a9f72e3 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:28.846148 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c5fcf-048f-4761-8f1b-8fe81c1ef493 service cinder] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:28.848818 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b29f74c0-cd96-406e-a521-d2a88a9f72e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:28.929558 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:28.963035 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f74c0-cd96-406e-a521-d2a88a9f72e3 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:28.972783 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb85aa3-c8b7-4acd-8143-042525d3170e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:28.973043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e5fe0-7ccf-4c69-9658-a780e75e9d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:28.974020 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb85aa3-c8b7-4acd-8143-042525d3170e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:28.974187 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e5fe0-7ccf-4c69-9658-a780e75e9d5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:28.974491 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e5fe0-7ccf-4c69-9658-a780e75e9d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:28.974660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb85aa3-c8b7-4acd-8143-042525d3170e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:28.974966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e5fe0-7ccf-4c69-9658-a780e75e9d5f admin admin] PATH_INFO: `/v3/projects/a73dce8e992b4df6af7887cfa64e8c5b` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:28.975144 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb85aa3-c8b7-4acd-8143-042525d3170e admin admin] PATH_INFO: `/v3/users/5879ec142a6a46df8eec467a70a5be33` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:28.976595 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb85aa3-c8b7-4acd-8143-042525d3170e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5879ec142a6a46df8eec467a70a5be33)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:28.978464 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e5fe0-7ccf-4c69-9658-a780e75e9d5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a73dce8e992b4df6af7887cfa64e8c5b)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:28.979207 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb85aa3-c8b7-4acd-8143-042525d3170e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:28.981931 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e5fe0-7ccf-4c69-9658-a780e75e9d5f admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:29.052975 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8fb85aa3-c8b7-4acd-8143-042525d3170e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5879ec142a6a46df8eec467a70a5be33'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:29.063647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8fb85aa3-c8b7-4acd-8143-042525d3170e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5879ec142a6a46df8eec467a70a5be33'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:29.064927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8fb85aa3-c8b7-4acd-8143-042525d3170e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5879ec142a6a46df8eec467a70a5be33'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:29.066295 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8fb85aa3-c8b7-4acd-8143-042525d3170e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5879ec142a6a46df8eec467a70a5be33'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:29.087116 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8fb85aa3-c8b7-4acd-8143-042525d3170e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5879ec142a6a46df8eec467a70a5be33'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:29.091678 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8a6e5fe0-7ccf-4c69-9658-a780e75e9d5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a73dce8e992b4df6af7887cfa64e8c5b', 'request_id': 'req-8a6e5fe0-7ccf-4c69-9658-a780e75e9d5f'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:29.107321 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8a6e5fe0-7ccf-4c69-9658-a780e75e9d5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a73dce8e992b4df6af7887cfa64e8c5b', 'request_id': 'req-8a6e5fe0-7ccf-4c69-9658-a780e75e9d5f'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:29.114458 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 946/1862] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:28 2022] DELETE /identity/v3/users/5879ec142a6a46df8eec467a70a5be33 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:29.120227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8a6e5fe0-7ccf-4c69-9658-a780e75e9d5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a73dce8e992b4df6af7887cfa64e8c5b', 'request_id': 'req-8a6e5fe0-7ccf-4c69-9658-a780e75e9d5f'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:29.130582 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:29.137988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8a6e5fe0-7ccf-4c69-9658-a780e75e9d5f admin admin] The token cache is being invalidate because project a73dce8e992b4df6af7887cfa64e8c5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:29.139448 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8a6e5fe0-7ccf-4c69-9658-a780e75e9d5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:29.143291 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8a6e5fe0-7ccf-4c69-9658-a780e75e9d5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:29.147301 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 917/1863] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:28 2022] DELETE /identity/v3/projects/a73dce8e992b4df6af7887cfa64e8c5b => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:19:29.156606 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e213e7-9919-4cbf-9feb-64c47185c14f None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:29.157916 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e213e7-9919-4cbf-9feb-64c47185c14f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:29.158113 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e213e7-9919-4cbf-9feb-64c47185c14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:29.231595 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-08e213e7-9919-4cbf-9feb-64c47185c14f None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:29.233626 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-08e213e7-9919-4cbf-9feb-64c47185c14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:29.483983 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 947/1864] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:19:29.512836 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e134b-a244-4821-a975-964136182f7a None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:29.513351 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e134b-a244-4821-a975-964136182f7a None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:29.513828 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e134b-a244-4821-a975-964136182f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:29.600095 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-307e134b-a244-4821-a975-964136182f7a None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:29.605076 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-307e134b-a244-4821-a975-964136182f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:29.644426 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 918/1865] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:29.650658 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e213e7-9919-4cbf-9feb-64c47185c14f None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:29.668710 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e213e7-9919-4cbf-9feb-64c47185c14f None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:29.683838 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6e4ff-2d1e-403c-afa6-aacd37e6963b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:29.684531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6e4ff-2d1e-403c-afa6-aacd37e6963b admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:29.684812 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6e4ff-2d1e-403c-afa6-aacd37e6963b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:29.686791 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6e4ff-2d1e-403c-afa6-aacd37e6963b admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:29.687671 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad6e4ff-2d1e-403c-afa6-aacd37e6963b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:19:29.688129 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad6e4ff-2d1e-403c-afa6-aacd37e6963b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:29.693392 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad6e4ff-2d1e-403c-afa6-aacd37e6963b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:29.725657 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 948/1866] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:19:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:19:29.729004 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e134b-a244-4821-a975-964136182f7a None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:29.754325 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e134b-a244-4821-a975-964136182f7a None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:29.774670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae17f72-b8c0-427e-9c74-5c7d30956ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:29.777744 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae17f72-b8c0-427e-9c74-5c7d30956ddd admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:29.778493 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae17f72-b8c0-427e-9c74-5c7d30956ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:29.779201 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae17f72-b8c0-427e-9c74-5c7d30956ddd admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:29.780632 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae17f72-b8c0-427e-9c74-5c7d30956ddd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:29.789318 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae17f72-b8c0-427e-9c74-5c7d30956ddd admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:29.893789 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 919/1867] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:29 2022] POST /identity/v3/projects => generated 367 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:29.896842 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6e4ff-2d1e-403c-afa6-aacd37e6963b admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:29.914702 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6e4ff-2d1e-403c-afa6-aacd37e6963b admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:29.998637 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:30.014353 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e42c46-8144-4078-b4a4-14b6e27dcb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:30.015678 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e42c46-8144-4078-b4a4-14b6e27dcb65 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:30.016029 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e42c46-8144-4078-b4a4-14b6e27dcb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:30.016446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e42c46-8144-4078-b4a4-14b6e27dcb65 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:30.017911 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e42c46-8144-4078-b4a4-14b6e27dcb65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:30.020673 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e42c46-8144-4078-b4a4-14b6e27dcb65 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:30.061861 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 949/1868] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:29 2022] POST /identity/v3/users => generated 284 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:30.062798 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae17f72-b8c0-427e-9c74-5c7d30956ddd admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:30.086418 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae17f72-b8c0-427e-9c74-5c7d30956ddd admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:30.096603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda149e1-ef35-4ac7-acd6-f0169380c297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:30.098508 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda149e1-ef35-4ac7-acd6-f0169380c297 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:30.099910 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda149e1-ef35-4ac7-acd6-f0169380c297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:30.100204 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda149e1-ef35-4ac7-acd6-f0169380c297 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:30.102698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda149e1-ef35-4ac7-acd6-f0169380c297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:30.109474 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda149e1-ef35-4ac7-acd6-f0169380c297 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:30.137074 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 920/1869] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:30.143955 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e42c46-8144-4078-b4a4-14b6e27dcb65 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:30.158138 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f8e42c46-8144-4078-b4a4-14b6e27dcb65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:30.163224 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda149e1-ef35-4ac7-acd6-f0169380c297 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:30.187989 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e42c46-8144-4078-b4a4-14b6e27dcb65 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:30.191595 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda149e1-ef35-4ac7-acd6-f0169380c297 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:30.199482 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8288b9-6630-4741-9582-19eff5f32981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:30.200416 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8288b9-6630-4741-9582-19eff5f32981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:30.200853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8288b9-6630-4741-9582-19eff5f32981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:30.203364 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8288b9-6630-4741-9582-19eff5f32981 admin admin] PATH_INFO: `/v3/projects/7fa438bda19443c5bcd640af2b89736f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:30.208293 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2b10f-9d4f-4b1e-87fb-83ccc9a2e4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:30.209298 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2b10f-9d4f-4b1e-87fb-83ccc9a2e4b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:30.209780 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2b10f-9d4f-4b1e-87fb-83ccc9a2e4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:30.210069 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2b10f-9d4f-4b1e-87fb-83ccc9a2e4b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:30.211157 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc2b10f-9d4f-4b1e-87fb-83ccc9a2e4b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:30.214259 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc2b10f-9d4f-4b1e-87fb-83ccc9a2e4b7 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:30.215944 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8288b9-6630-4741-9582-19eff5f32981 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fa438bda19443c5bcd640af2b89736f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:30.219278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8288b9-6630-4741-9582-19eff5f32981 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:30.259596 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 921/1870] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:30.260623 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2b10f-9d4f-4b1e-87fb-83ccc9a2e4b7 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:30.302231 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2b10f-9d4f-4b1e-87fb-83ccc9a2e4b7 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:30.325076 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc88bf-8856-44ac-be17-fa5d09c28d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:30.330154 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc88bf-8856-44ac-be17-fa5d09c28d41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:30.334916 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc88bf-8856-44ac-be17-fa5d09c28d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:30.335461 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc88bf-8856-44ac-be17-fa5d09c28d41 admin admin] PATH_INFO: `/v3/projects/849b392010134a59aaa39da54fc15152/users/d4c723489b5643a9a60a34c513b265c0/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:30.386421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc88bf-8856-44ac-be17-fa5d09c28d41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=849b392010134a59aaa39da54fc15152, user_id=d4c723489b5643a9a60a34c513b265c0, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:30.387207 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4b8288b9-6630-4741-9582-19eff5f32981 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fa438bda19443c5bcd640af2b89736f', 'request_id': 'req-4b8288b9-6630-4741-9582-19eff5f32981'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:30.395615 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc88bf-8856-44ac-be17-fa5d09c28d41 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:30.404436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4b8288b9-6630-4741-9582-19eff5f32981 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fa438bda19443c5bcd640af2b89736f', 'request_id': 'req-4b8288b9-6630-4741-9582-19eff5f32981'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:30.433186 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4b8288b9-6630-4741-9582-19eff5f32981 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fa438bda19443c5bcd640af2b89736f', 'request_id': 'req-4b8288b9-6630-4741-9582-19eff5f32981'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:30.437325 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 922/1871] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:30 2022] PUT /identity/v3/projects/849b392010134a59aaa39da54fc15152/users/d4c723489b5643a9a60a34c513b265c0/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:30.454384 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55ec59-6a36-4e55-82a0-1adfb2f04fe3 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:30.455306 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55ec59-6a36-4e55-82a0-1adfb2f04fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:30.456129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55ec59-6a36-4e55-82a0-1adfb2f04fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:30.480517 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4b8288b9-6630-4741-9582-19eff5f32981 admin admin] The token cache is being invalidate because project 7fa438bda19443c5bcd640af2b89736f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:30.481468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4b8288b9-6630-4741-9582-19eff5f32981 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:30.484779 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4b8288b9-6630-4741-9582-19eff5f32981 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:30.494843 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 950/1872] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:30 2022] DELETE /identity/v3/projects/7fa438bda19443c5bcd640af2b89736f => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:19:30.503765 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2687ea6-2db7-42f3-bc94-b506cb869959 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:30.503765 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2687ea6-2db7-42f3-bc94-b506cb869959 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:30.504095 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2687ea6-2db7-42f3-bc94-b506cb869959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:30.585256 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-bc55ec59-6a36-4e55-82a0-1adfb2f04fe3 None None] MFA Rules not processed for user `d4c723489b5643a9a60a34c513b265c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:30.587592 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-bc55ec59-6a36-4e55-82a0-1adfb2f04fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:30.608848 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-d2687ea6-2db7-42f3-bc94-b506cb869959 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:30.611815 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d2687ea6-2db7-42f3-bc94-b506cb869959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:30.746333 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 951/1873] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:30.755021 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a09f1-1110-4808-bb16-fcb605ff8e67 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:30.755310 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a09f1-1110-4808-bb16-fcb605ff8e67 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:30.755851 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a09f1-1110-4808-bb16-fcb605ff8e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:30.817930 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-173a09f1-1110-4808-bb16-fcb605ff8e67 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:30.821958 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-173a09f1-1110-4808-bb16-fcb605ff8e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:30.845663 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 923/1874] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:19:30.845663 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc55ec59-6a36-4e55-82a0-1adfb2f04fe3 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:30.858094 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-bc55ec59-6a36-4e55-82a0-1adfb2f04fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:30.861607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 952/1875] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:30.880165 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a09f1-1110-4808-bb16-fcb605ff8e67 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:30.909340 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc55ec59-6a36-4e55-82a0-1adfb2f04fe3 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:30.917158 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a09f1-1110-4808-bb16-fcb605ff8e67 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:30.922646 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da2dd24-5325-4b4c-b8c4-a65850429e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:30.923990 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da2dd24-5325-4b4c-b8c4-a65850429e39 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:30.924266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da2dd24-5325-4b4c-b8c4-a65850429e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:30.924535 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da2dd24-5325-4b4c-b8c4-a65850429e39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:30.928252 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da2dd24-5325-4b4c-b8c4-a65850429e39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:30.933006 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da2dd24-5325-4b4c-b8c4-a65850429e39 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:30.947426 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5159a6-b15b-4ef6-8dc4-3a315afb5b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:30.948142 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5159a6-b15b-4ef6-8dc4-3a315afb5b0b admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:30.948423 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5159a6-b15b-4ef6-8dc4-3a315afb5b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:30.948703 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5159a6-b15b-4ef6-8dc4-3a315afb5b0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:30.949421 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5159a6-b15b-4ef6-8dc4-3a315afb5b0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:19:30.950418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5159a6-b15b-4ef6-8dc4-3a315afb5b0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:30.951820 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5159a6-b15b-4ef6-8dc4-3a315afb5b0b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:30.997234 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 953/1876] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:19:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:19:31.003148 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5159a6-b15b-4ef6-8dc4-3a315afb5b0b admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:31.020177 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5159a6-b15b-4ef6-8dc4-3a315afb5b0b admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:31.044798 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab2186-f741-4b92-a3c2-d6a579a9baf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:31.045835 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab2186-f741-4b92-a3c2-d6a579a9baf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:31.046316 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab2186-f741-4b92-a3c2-d6a579a9baf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:31.046779 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab2186-f741-4b92-a3c2-d6a579a9baf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:31.047667 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab2186-f741-4b92-a3c2-d6a579a9baf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:31.051268 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab2186-f741-4b92-a3c2-d6a579a9baf0 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:31.073290 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 924/1877] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:30 2022] POST /identity/v3/projects => generated 367 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:31.077565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da2dd24-5325-4b4c-b8c4-a65850429e39 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:31.119355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 954/1878] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:31 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:19:31.124817 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab2186-f741-4b92-a3c2-d6a579a9baf0 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:31.204735 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:31.285173 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab2186-f741-4b92-a3c2-d6a579a9baf0 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:31.302559 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0f7fe-2d74-4c0f-b6aa-b1f086910e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:31.306876 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0f7fe-2d74-4c0f-b6aa-b1f086910e7f admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:31.307567 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0f7fe-2d74-4c0f-b6aa-b1f086910e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:31.308230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0f7fe-2d74-4c0f-b6aa-b1f086910e7f admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:31.309125 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0f7fe-2d74-4c0f-b6aa-b1f086910e7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:31.312885 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0f7fe-2d74-4c0f-b6aa-b1f086910e7f admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:31.367027 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:31.396755 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 955/1879] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:31 2022] POST /identity/v3/users => generated 290 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:31.400973 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0f7fe-2d74-4c0f-b6aa-b1f086910e7f admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:31.426135 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da2dd24-5325-4b4c-b8c4-a65850429e39 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:31.428038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0f7fe-2d74-4c0f-b6aa-b1f086910e7f admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:31.446076 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df66c4-48f1-49ad-89fe-431fe5653f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:31.449892 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df66c4-48f1-49ad-89fe-431fe5653f90 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:31.451060 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df66c4-48f1-49ad-89fe-431fe5653f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:31.451742 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8710b-1608-4fca-9b3c-c5f1ba9efd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:31.454878 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df66c4-48f1-49ad-89fe-431fe5653f90 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:31.455600 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8710b-1608-4fca-9b3c-c5f1ba9efd23 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:31.460101 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df66c4-48f1-49ad-89fe-431fe5653f90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:31.461117 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8710b-1608-4fca-9b3c-c5f1ba9efd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:31.462273 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8710b-1608-4fca-9b3c-c5f1ba9efd23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:31.464222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df66c4-48f1-49ad-89fe-431fe5653f90 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:31.475558 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d8710b-1608-4fca-9b3c-c5f1ba9efd23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:31.475558 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d8710b-1608-4fca-9b3c-c5f1ba9efd23 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:31.514510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 956/1880] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:31.515579 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 925/1881] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:31 2022] POST /identity/v3/users => generated 290 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:31.520989 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8710b-1608-4fca-9b3c-c5f1ba9efd23 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:31.523244 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df66c4-48f1-49ad-89fe-431fe5653f90 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:31.544522 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df66c4-48f1-49ad-89fe-431fe5653f90 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:31.554056 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf126c0-5897-4cf7-8eeb-aa4411f543ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:31.556643 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf126c0-5897-4cf7-8eeb-aa4411f543ee admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:31.557471 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf126c0-5897-4cf7-8eeb-aa4411f543ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:31.558097 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8710b-1608-4fca-9b3c-c5f1ba9efd23 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:31.558866 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf126c0-5897-4cf7-8eeb-aa4411f543ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:31.560560 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf126c0-5897-4cf7-8eeb-aa4411f543ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:31.564547 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf126c0-5897-4cf7-8eeb-aa4411f543ee admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:31.573681 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4473d60-2d0c-46aa-88bd-42f411d0cab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:31.577986 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4473d60-2d0c-46aa-88bd-42f411d0cab1 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:31.578278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4473d60-2d0c-46aa-88bd-42f411d0cab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:31.578427 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4473d60-2d0c-46aa-88bd-42f411d0cab1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:31.583651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4473d60-2d0c-46aa-88bd-42f411d0cab1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:31.587808 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4473d60-2d0c-46aa-88bd-42f411d0cab1 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:31.596589 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 926/1882] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:19:31.601117 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf126c0-5897-4cf7-8eeb-aa4411f543ee admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:31.617280 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 957/1883] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:19:31.622630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4473d60-2d0c-46aa-88bd-42f411d0cab1 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:31.637600 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf126c0-5897-4cf7-8eeb-aa4411f543ee admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:31.648541 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4473d60-2d0c-46aa-88bd-42f411d0cab1 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:31.673082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2291f6-9a95-40d6-8017-f71a01b5c6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:31.673642 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2291f6-9a95-40d6-8017-f71a01b5c6b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:31.673925 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2291f6-9a95-40d6-8017-f71a01b5c6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:31.679587 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2291f6-9a95-40d6-8017-f71a01b5c6b5 admin admin] PATH_INFO: `/v3/projects/7558aa222641481f83c138ad4dca7342/users/7b89bcae79854d869c8bf39c23c8f700/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:31.689685 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6380077b-2feb-43a6-a637-78c0f16b2925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:31.706790 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6380077b-2feb-43a6-a637-78c0f16b2925 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:31.710228 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6380077b-2feb-43a6-a637-78c0f16b2925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:31.711237 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6380077b-2feb-43a6-a637-78c0f16b2925 admin admin] PATH_INFO: `/v3/projects/cfaa5c42b3844316a71fb08d24735f04/users/2c8370ecd5604849a9d9590ce536bb87/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:31.741978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2291f6-9a95-40d6-8017-f71a01b5c6b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7558aa222641481f83c138ad4dca7342, user_id=7b89bcae79854d869c8bf39c23c8f700, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:31.750248 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2291f6-9a95-40d6-8017-f71a01b5c6b5 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:31.755918 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6380077b-2feb-43a6-a637-78c0f16b2925 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfaa5c42b3844316a71fb08d24735f04, user_id=2c8370ecd5604849a9d9590ce536bb87, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:31.763348 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6380077b-2feb-43a6-a637-78c0f16b2925 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:31.786498 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 958/1884] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:31 2022] PUT /identity/v3/projects/7558aa222641481f83c138ad4dca7342/users/7b89bcae79854d869c8bf39c23c8f700/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:31.793636 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2291f6-9a95-40d6-8017-f71a01b5c6b5 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:31.804277 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 927/1885] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:31 2022] PUT /identity/v3/projects/cfaa5c42b3844316a71fb08d24735f04/users/2c8370ecd5604849a9d9590ce536bb87/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:31.832265 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b506dae-93ee-4d85-a269-e77ac861da78 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:31.832711 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b506dae-93ee-4d85-a269-e77ac861da78 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:31.832992 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b506dae-93ee-4d85-a269-e77ac861da78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:31.905747 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-6b506dae-93ee-4d85-a269-e77ac861da78 None None] MFA Rules not processed for user `2c8370ecd5604849a9d9590ce536bb87`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:31.908959 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-6b506dae-93ee-4d85-a269-e77ac861da78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:31.952440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2291f6-9a95-40d6-8017-f71a01b5c6b5 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:32.077975 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 928/1886] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:32.082895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b506dae-93ee-4d85-a269-e77ac861da78 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:32.306791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b506dae-93ee-4d85-a269-e77ac861da78 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:32.323888 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f3942-6188-439e-b8ff-23c1edd8a407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:32.324715 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f3942-6188-439e-b8ff-23c1edd8a407 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:32.324891 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f3942-6188-439e-b8ff-23c1edd8a407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:32.325188 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f3942-6188-439e-b8ff-23c1edd8a407 admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:32.325756 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f3942-6188-439e-b8ff-23c1edd8a407 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:32.327738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f3942-6188-439e-b8ff-23c1edd8a407 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:32.335966 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8658a-eb8e-402e-8c27-ccf02b850258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:32.338645 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8658a-eb8e-402e-8c27-ccf02b850258 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:32.347413 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8658a-eb8e-402e-8c27-ccf02b850258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:32.347900 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8658a-eb8e-402e-8c27-ccf02b850258 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:32.350690 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8658a-eb8e-402e-8c27-ccf02b850258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:32.361274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8658a-eb8e-402e-8c27-ccf02b850258 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:32.391295 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 929/1887] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:32 2022] POST /identity/v3/projects => generated 383 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:19:32.395207 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f3942-6188-439e-b8ff-23c1edd8a407 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:32.403372 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 959/1888] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 612 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:32.405025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8658a-eb8e-402e-8c27-ccf02b850258 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:32.451673 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8658a-eb8e-402e-8c27-ccf02b850258 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:32.458167 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:32.494303 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:32.580359 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f3942-6188-439e-b8ff-23c1edd8a407 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:32.604449 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb3fd1-cbea-4ae1-9f44-35e1d8fe814c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:32.606353 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb3fd1-cbea-4ae1-9f44-35e1d8fe814c admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:32.609171 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb3fd1-cbea-4ae1-9f44-35e1d8fe814c admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:32.622419 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb3fd1-cbea-4ae1-9f44-35e1d8fe814c admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:32.623950 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb3fd1-cbea-4ae1-9f44-35e1d8fe814c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:32.632531 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb3fd1-cbea-4ae1-9f44-35e1d8fe814c admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:32.707410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 930/1889] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:32 2022] POST /identity/v3/users => generated 298 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:32.709759 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0c5e1-131a-4329-83de-272d70efc202 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:32.710486 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0c5e1-131a-4329-83de-272d70efc202 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:32.711017 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0c5e1-131a-4329-83de-272d70efc202 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:32.715385 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 931/1890] 10.222.0.16 () {54 vars in 921 bytes} [Mon Sep 5 12:19:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:19:32.717176 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0c5e1-131a-4329-83de-272d70efc202 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:32.742565 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761ef56-004c-4587-8b83-96fa07eff12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:32.744139 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761ef56-004c-4587-8b83-96fa07eff12c admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:32.744857 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761ef56-004c-4587-8b83-96fa07eff12c admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:32.745571 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761ef56-004c-4587-8b83-96fa07eff12c admin admin] PATH_INFO: `/v3/projects/7558aa222641481f83c138ad4dca7342/users/7b89bcae79854d869c8bf39c23c8f700/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:32.749397 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0c5e1-131a-4329-83de-272d70efc202 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:32.756977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7761ef56-004c-4587-8b83-96fa07eff12c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7558aa222641481f83c138ad4dca7342, user_id=7b89bcae79854d869c8bf39c23c8f700, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:32.758867 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf59e7a-07b1-403a-b3f3-3e5adaf2a764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:32.759930 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf59e7a-07b1-403a-b3f3-3e5adaf2a764 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:32.760388 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf59e7a-07b1-403a-b3f3-3e5adaf2a764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:32.760848 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf59e7a-07b1-403a-b3f3-3e5adaf2a764 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:32.761996 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf59e7a-07b1-403a-b3f3-3e5adaf2a764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:32.765256 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf59e7a-07b1-403a-b3f3-3e5adaf2a764 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:32.772199 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7761ef56-004c-4587-8b83-96fa07eff12c admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:32.802680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 932/1892] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:32 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:32.805658 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d1281-9950-491d-8769-1f579d825bcb None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:32.807166 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d1281-9950-491d-8769-1f579d825bcb None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:32.808070 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d1281-9950-491d-8769-1f579d825bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:32.812412 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 960/1892] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:32 2022] PUT /identity/v3/projects/7558aa222641481f83c138ad4dca7342/users/7b89bcae79854d869c8bf39c23c8f700/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:32.817477 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b72c0-b489-49d1-bbe1-c2a7c187e165 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:32.817980 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b72c0-b489-49d1-bbe1-c2a7c187e165 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:32.818431 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b72c0-b489-49d1-bbe1-c2a7c187e165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:32.848980 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-547d1281-9950-491d-8769-1f579d825bcb None None] MFA Rules not processed for user `7cda290418504f0db85d665479a05353`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:32.851289 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-547d1281-9950-491d-8769-1f579d825bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:32.865539 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-bf1b72c0-b489-49d1-bbe1-c2a7c187e165 None None] MFA Rules not processed for user `7b89bcae79854d869c8bf39c23c8f700`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:32.867177 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-bf1b72c0-b489-49d1-bbe1-c2a7c187e165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:33.059113 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 933/1893] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Sep 5 12:19:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:33.062609 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d1281-9950-491d-8769-1f579d825bcb None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:33.116949 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 961/1894] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:32 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:33.118986 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b72c0-b489-49d1-bbe1-c2a7c187e165 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:33.122899 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-bf1b72c0-b489-49d1-bbe1-c2a7c187e165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:33.174639 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b72c0-b489-49d1-bbe1-c2a7c187e165 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:33.200550 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d734e-01d8-4e21-a818-dba7e9003338 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:33.201218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d734e-01d8-4e21-a818-dba7e9003338 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:33.201473 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d734e-01d8-4e21-a818-dba7e9003338 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:33.201746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d734e-01d8-4e21-a818-dba7e9003338 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:33.245123 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d734e-01d8-4e21-a818-dba7e9003338 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:33.246760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d734e-01d8-4e21-a818-dba7e9003338 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:33.276943 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 962/1895] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:33.288625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d734e-01d8-4e21-a818-dba7e9003338 service nova] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:33.292058 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4a2d734e-01d8-4e21-a818-dba7e9003338 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:33.374938 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d1281-9950-491d-8769-1f579d825bcb None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:33.402347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d734e-01d8-4e21-a818-dba7e9003338 service nova] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:33.407058 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea649bda-fa9e-4ea9-8eda-279c9ab02d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:33.408380 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562d63e-64a9-4ba1-827e-eb73a5586ede service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:33.409304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562d63e-64a9-4ba1-827e-eb73a5586ede service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:33.409894 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562d63e-64a9-4ba1-827e-eb73a5586ede service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:33.410329 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562d63e-64a9-4ba1-827e-eb73a5586ede service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:33.412667 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea649bda-fa9e-4ea9-8eda-279c9ab02d3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:33.413419 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea649bda-fa9e-4ea9-8eda-279c9ab02d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:33.418269 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea649bda-fa9e-4ea9-8eda-279c9ab02d3e admin admin] PATH_INFO: `/v3/projects/8721adad308f486892506dc546d55bad/users/ab485ef5613e447f894358551986fd1a/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:33.426952 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f562d63e-64a9-4ba1-827e-eb73a5586ede service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:33.428973 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f562d63e-64a9-4ba1-827e-eb73a5586ede service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:33.446490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 963/1896] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:33 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:33.560987 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea649bda-fa9e-4ea9-8eda-279c9ab02d3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8721adad308f486892506dc546d55bad, user_id=ab485ef5613e447f894358551986fd1a, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:33.575321 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea649bda-fa9e-4ea9-8eda-279c9ab02d3e admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:33.631822 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 934/1897] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:33 2022] PUT /identity/v3/projects/8721adad308f486892506dc546d55bad/users/ab485ef5613e447f894358551986fd1a/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:33.633242 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562d63e-64a9-4ba1-827e-eb73a5586ede service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:33.650816 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562d63e-64a9-4ba1-827e-eb73a5586ede service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:33.692750 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:33.798861 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794bb31-2920-420b-a7eb-0d6c1aec3e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:33.800653 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794bb31-2920-420b-a7eb-0d6c1aec3e64 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:33.804250 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794bb31-2920-420b-a7eb-0d6c1aec3e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:33.805166 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794bb31-2920-420b-a7eb-0d6c1aec3e64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:33.806026 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3794bb31-2920-420b-a7eb-0d6c1aec3e64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:33.811281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3794bb31-2920-420b-a7eb-0d6c1aec3e64 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:33.840952 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 964/1898] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:33.848743 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea649bda-fa9e-4ea9-8eda-279c9ab02d3e admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:33.878564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794bb31-2920-420b-a7eb-0d6c1aec3e64 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:33.880977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3794bb31-2920-420b-a7eb-0d6c1aec3e64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:33.882093 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea649bda-fa9e-4ea9-8eda-279c9ab02d3e admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:33.902719 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7e0bb-0755-4800-8c81-d775a1240f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:33.905006 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7e0bb-0755-4800-8c81-d775a1240f41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:33.905522 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7e0bb-0755-4800-8c81-d775a1240f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:33.906086 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7e0bb-0755-4800-8c81-d775a1240f41 admin admin] PATH_INFO: `/v3/projects/8721adad308f486892506dc546d55bad/users/ab485ef5613e447f894358551986fd1a/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:33.917808 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f7e0bb-0755-4800-8c81-d775a1240f41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8721adad308f486892506dc546d55bad, user_id=ab485ef5613e447f894358551986fd1a, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:33.936944 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f7e0bb-0755-4800-8c81-d775a1240f41 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:33.966725 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:34.003224 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 935/1899] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:33 2022] PUT /identity/v3/projects/8721adad308f486892506dc546d55bad/users/ab485ef5613e447f894358551986fd1a/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:34.003948 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794bb31-2920-420b-a7eb-0d6c1aec3e64 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:34.006491 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f82c72-0822-4cc3-806f-d2c88bcd9b62 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:34.006491 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f82c72-0822-4cc3-806f-d2c88bcd9b62 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:34.006842 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f82c72-0822-4cc3-806f-d2c88bcd9b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:34.143737 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-58f82c72-0822-4cc3-806f-d2c88bcd9b62 None None] MFA Rules not processed for user `ab485ef5613e447f894358551986fd1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:34.145886 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-58f82c72-0822-4cc3-806f-d2c88bcd9b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:34.183963 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d2715b-aeca-4d4f-89c4-9f19140d1a60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:34.190858 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2715b-aeca-4d4f-89c4-9f19140d1a60 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:34.191360 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2715b-aeca-4d4f-89c4-9f19140d1a60 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:34.191857 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2715b-aeca-4d4f-89c4-9f19140d1a60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:34.337881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 936/1900] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:34 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:34.366957 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d2715b-aeca-4d4f-89c4-9f19140d1a60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:34.383600 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d2715b-aeca-4d4f-89c4-9f19140d1a60 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:34.389619 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 965/1901] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:19:34.565071 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f82c72-0822-4cc3-806f-d2c88bcd9b62 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:34.581054 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f82c72-0822-4cc3-806f-d2c88bcd9b62 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:34.587430 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70cdfe-700f-4b2d-a42f-b11d61d1a5cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:34.589827 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70cdfe-700f-4b2d-a42f-b11d61d1a5cb service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:34.590495 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70cdfe-700f-4b2d-a42f-b11d61d1a5cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:34.590955 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70cdfe-700f-4b2d-a42f-b11d61d1a5cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:34.611274 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70cdfe-700f-4b2d-a42f-b11d61d1a5cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:34.615332 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70cdfe-700f-4b2d-a42f-b11d61d1a5cb service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:34.651781 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 937/1902] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:34 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:19:35.504422 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d2715b-aeca-4d4f-89c4-9f19140d1a60 service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:35.507230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-92d2715b-aeca-4d4f-89c4-9f19140d1a60 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:35.510885 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70cdfe-700f-4b2d-a42f-b11d61d1a5cb service cinder] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:35.525945 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70cdfe-700f-4b2d-a42f-b11d61d1a5cb service cinder] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:35.537678 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528b5c0-cb53-4649-9025-5fba64e975bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:35.538527 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528b5c0-cb53-4649-9025-5fba64e975bd service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:35.538907 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528b5c0-cb53-4649-9025-5fba64e975bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:35.539280 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528b5c0-cb53-4649-9025-5fba64e975bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:35.593744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:35.679942 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d2715b-aeca-4d4f-89c4-9f19140d1a60 service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:35.687048 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a528b5c0-cb53-4649-9025-5fba64e975bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:35.690339 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a528b5c0-cb53-4649-9025-5fba64e975bd service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:35.692469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718196f8-64d3-4fc7-9c4a-3e499792796b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:35.693308 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718196f8-64d3-4fc7-9c4a-3e499792796b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:35.693682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718196f8-64d3-4fc7-9c4a-3e499792796b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:35.694049 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718196f8-64d3-4fc7-9c4a-3e499792796b admin admin] PATH_INFO: `/v3/users/62ee3f143a9e4bfa852c7e794dd7d990` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:35.697886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718196f8-64d3-4fc7-9c4a-3e499792796b admin admin] RBAC: Authorizing `identity:delete_user(user_id=62ee3f143a9e4bfa852c7e794dd7d990)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:35.703276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718196f8-64d3-4fc7-9c4a-3e499792796b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:35.713248 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 938/1903] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:35.765960 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528b5c0-cb53-4649-9025-5fba64e975bd service cinder] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:35.769588 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-a528b5c0-cb53-4649-9025-5fba64e975bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:35.798235 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528b5c0-cb53-4649-9025-5fba64e975bd service cinder] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:35.805819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86e2dd-eea5-40c7-b6ac-09387a559101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:35.806642 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86e2dd-eea5-40c7-b6ac-09387a559101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:35.807027 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86e2dd-eea5-40c7-b6ac-09387a559101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:35.807402 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86e2dd-eea5-40c7-b6ac-09387a559101 admin admin] PATH_INFO: `/v3/users/7b07b31e3d384a0e81039a7e99d405a0` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:35.808872 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d86e2dd-eea5-40c7-b6ac-09387a559101 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b07b31e3d384a0e81039a7e99d405a0)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:35.811007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d86e2dd-eea5-40c7-b6ac-09387a559101 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:35.856053 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-718196f8-64d3-4fc7-9c4a-3e499792796b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62ee3f143a9e4bfa852c7e794dd7d990'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:35.871706 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-718196f8-64d3-4fc7-9c4a-3e499792796b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62ee3f143a9e4bfa852c7e794dd7d990'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:35.873056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-718196f8-64d3-4fc7-9c4a-3e499792796b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62ee3f143a9e4bfa852c7e794dd7d990'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:35.874384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-718196f8-64d3-4fc7-9c4a-3e499792796b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62ee3f143a9e4bfa852c7e794dd7d990'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:35.902032 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-718196f8-64d3-4fc7-9c4a-3e499792796b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62ee3f143a9e4bfa852c7e794dd7d990'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:35.905309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-1d86e2dd-eea5-40c7-b6ac-09387a559101 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b07b31e3d384a0e81039a7e99d405a0'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:35.920876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-1d86e2dd-eea5-40c7-b6ac-09387a559101 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b07b31e3d384a0e81039a7e99d405a0'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:35.922684 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-1d86e2dd-eea5-40c7-b6ac-09387a559101 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b07b31e3d384a0e81039a7e99d405a0'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:35.936682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 966/1904] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:35 2022] DELETE /identity/v3/users/62ee3f143a9e4bfa852c7e794dd7d990 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:35.939394 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:35.940133 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89cd83-4a8b-49c4-a1bd-2dffcfa3c449 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:35.941085 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89cd83-4a8b-49c4-a1bd-2dffcfa3c449 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:35.941967 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89cd83-4a8b-49c4-a1bd-2dffcfa3c449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:35.950709 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-1d86e2dd-eea5-40c7-b6ac-09387a559101 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b07b31e3d384a0e81039a7e99d405a0'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:35.973361 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-1d86e2dd-eea5-40c7-b6ac-09387a559101 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b07b31e3d384a0e81039a7e99d405a0'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:35.988315 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 939/1905] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:35 2022] DELETE /identity/v3/users/7b07b31e3d384a0e81039a7e99d405a0 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:19:35.993907 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-da89cd83-4a8b-49c4-a1bd-2dffcfa3c449 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:35.995887 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c6eb3-0adb-4c01-904d-8e66bc232214 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:35.996099 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-da89cd83-4a8b-49c4-a1bd-2dffcfa3c449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:35.996344 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c6eb3-0adb-4c01-904d-8e66bc232214 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:35.996907 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c6eb3-0adb-4c01-904d-8e66bc232214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:36.046967 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-3e9c6eb3-0adb-4c01-904d-8e66bc232214 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:36.050084 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-3e9c6eb3-0adb-4c01-904d-8e66bc232214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:36.119724 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 967/1906] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:36.137941 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89cd83-4a8b-49c4-a1bd-2dffcfa3c449 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:36.142228 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-da89cd83-4a8b-49c4-a1bd-2dffcfa3c449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:36.242473 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 940/1907] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:36.255886 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c6eb3-0adb-4c01-904d-8e66bc232214 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:36.258740 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-3e9c6eb3-0adb-4c01-904d-8e66bc232214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:36.340362 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:36.353765 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89cd83-4a8b-49c4-a1bd-2dffcfa3c449 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:36.366432 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917ac62-067b-4d9c-877a-5b2c83426b83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:36.367458 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917ac62-067b-4d9c-877a-5b2c83426b83 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:36.367954 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917ac62-067b-4d9c-877a-5b2c83426b83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:36.368398 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917ac62-067b-4d9c-877a-5b2c83426b83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:36.413491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917ac62-067b-4d9c-877a-5b2c83426b83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:36.415180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917ac62-067b-4d9c-877a-5b2c83426b83 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:36.455875 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 968/1908] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:19:36.503787 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c6eb3-0adb-4c01-904d-8e66bc232214 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:36.517402 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7bd7b-1e4b-451c-a384-2dab4671db9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:36.518085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7bd7b-1e4b-451c-a384-2dab4671db9e service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:36.518370 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7bd7b-1e4b-451c-a384-2dab4671db9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:36.518854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7bd7b-1e4b-451c-a384-2dab4671db9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:36.545632 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7bd7b-1e4b-451c-a384-2dab4671db9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:36.548784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7bd7b-1e4b-451c-a384-2dab4671db9e service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:36.585906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 941/1909] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:19:37.096933 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917ac62-067b-4d9c-877a-5b2c83426b83 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:37.099073 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7917ac62-067b-4d9c-877a-5b2c83426b83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:37.134922 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917ac62-067b-4d9c-877a-5b2c83426b83 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:37.145790 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7247719b-5e28-4ab4-80dc-0deb713ef331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:37.146855 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247719b-5e28-4ab4-80dc-0deb713ef331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:37.147252 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247719b-5e28-4ab4-80dc-0deb713ef331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:37.148129 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247719b-5e28-4ab4-80dc-0deb713ef331 admin admin] PATH_INFO: `/v3/projects/aa1b82105e884be5a2629c53dbf96fa5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:37.150402 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7247719b-5e28-4ab4-80dc-0deb713ef331 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa1b82105e884be5a2629c53dbf96fa5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:37.153891 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7247719b-5e28-4ab4-80dc-0deb713ef331 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:37.244786 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7247719b-5e28-4ab4-80dc-0deb713ef331 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa1b82105e884be5a2629c53dbf96fa5', 'request_id': 'req-7247719b-5e28-4ab4-80dc-0deb713ef331'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:37.260275 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7bd7b-1e4b-451c-a384-2dab4671db9e service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:37.263184 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-4cd7bd7b-1e4b-451c-a384-2dab4671db9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:37.266048 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7247719b-5e28-4ab4-80dc-0deb713ef331 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa1b82105e884be5a2629c53dbf96fa5', 'request_id': 'req-7247719b-5e28-4ab4-80dc-0deb713ef331'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:37.291565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7bd7b-1e4b-451c-a384-2dab4671db9e service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:37.306777 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7247719b-5e28-4ab4-80dc-0deb713ef331 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa1b82105e884be5a2629c53dbf96fa5', 'request_id': 'req-7247719b-5e28-4ab4-80dc-0deb713ef331'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:37.354957 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7247719b-5e28-4ab4-80dc-0deb713ef331 admin admin] The token cache is being invalidate because project aa1b82105e884be5a2629c53dbf96fa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:37.355390 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7247719b-5e28-4ab4-80dc-0deb713ef331 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:37.359285 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7247719b-5e28-4ab4-80dc-0deb713ef331 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:37.363092 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 969/1910] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:37 2022] DELETE /identity/v3/projects/aa1b82105e884be5a2629c53dbf96fa5 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:37.369074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7247719b-5e28-4ab4-80dc-0deb713ef331 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:37.372680 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7247719b-5e28-4ab4-80dc-0deb713ef331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:37.416533 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9fc06c-6640-482d-9ac7-22be69fd6d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:37.417582 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9fc06c-6640-482d-9ac7-22be69fd6d7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:37.418040 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9fc06c-6640-482d-9ac7-22be69fd6d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:37.418502 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9fc06c-6640-482d-9ac7-22be69fd6d7a admin admin] PATH_INFO: `/v3/projects/76d08d06cd7342f0a356160b2c371234` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:37.420683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9fc06c-6640-482d-9ac7-22be69fd6d7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=76d08d06cd7342f0a356160b2c371234)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:37.424906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9fc06c-6640-482d-9ac7-22be69fd6d7a admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:37.528750 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:37.569062 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-4f9fc06c-6640-482d-9ac7-22be69fd6d7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76d08d06cd7342f0a356160b2c371234', 'request_id': 'req-4f9fc06c-6640-482d-9ac7-22be69fd6d7a'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:37.589832 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-4f9fc06c-6640-482d-9ac7-22be69fd6d7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76d08d06cd7342f0a356160b2c371234', 'request_id': 'req-4f9fc06c-6640-482d-9ac7-22be69fd6d7a'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:37.605728 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7247719b-5e28-4ab4-80dc-0deb713ef331 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:37.610737 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-4f9fc06c-6640-482d-9ac7-22be69fd6d7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76d08d06cd7342f0a356160b2c371234', 'request_id': 'req-4f9fc06c-6640-482d-9ac7-22be69fd6d7a'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:37.638682 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-4f9fc06c-6640-482d-9ac7-22be69fd6d7a admin admin] The token cache is being invalidate because project 76d08d06cd7342f0a356160b2c371234 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:37.639164 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-4f9fc06c-6640-482d-9ac7-22be69fd6d7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:37.640330 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-4f9fc06c-6640-482d-9ac7-22be69fd6d7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:37.643832 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 942/1911] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:37 2022] DELETE /identity/v3/projects/76d08d06cd7342f0a356160b2c371234 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:37.651817 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9fc06c-6640-482d-9ac7-22be69fd6d7a admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:37.654275 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-4f9fc06c-6640-482d-9ac7-22be69fd6d7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:37.726527 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ff13c-6dc1-4049-a835-106ee4c24f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:37.727596 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ff13c-6dc1-4049-a835-106ee4c24f5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:37.728069 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ff13c-6dc1-4049-a835-106ee4c24f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:37.728516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ff13c-6dc1-4049-a835-106ee4c24f5b admin admin] PATH_INFO: `/v3/users/69a83cba7cf94579998f9e24edbd2adc` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:37.730528 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ff13c-6dc1-4049-a835-106ee4c24f5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=69a83cba7cf94579998f9e24edbd2adc)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:37.734186 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ff13c-6dc1-4049-a835-106ee4c24f5b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:37.755450 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9fc06c-6640-482d-9ac7-22be69fd6d7a admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:37.764376 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c1fc2-4774-4bd6-a17a-0ca1ffe0cb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:37.765596 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c1fc2-4774-4bd6-a17a-0ca1ffe0cb71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:37.766230 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c1fc2-4774-4bd6-a17a-0ca1ffe0cb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:37.767290 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c1fc2-4774-4bd6-a17a-0ca1ffe0cb71 admin admin] PATH_INFO: `/v3/users/0a73a00c3ce4496ebda509a7dde95b39` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:37.769543 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c1fc2-4774-4bd6-a17a-0ca1ffe0cb71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a73a00c3ce4496ebda509a7dde95b39)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:37.773361 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c1fc2-4774-4bd6-a17a-0ca1ffe0cb71 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:37.777024 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:37.827978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-926ff13c-6dc1-4049-a835-106ee4c24f5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69a83cba7cf94579998f9e24edbd2adc'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:37.843886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-926ff13c-6dc1-4049-a835-106ee4c24f5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69a83cba7cf94579998f9e24edbd2adc'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:37.843886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-926ff13c-6dc1-4049-a835-106ee4c24f5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69a83cba7cf94579998f9e24edbd2adc'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:37.845972 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-926ff13c-6dc1-4049-a835-106ee4c24f5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69a83cba7cf94579998f9e24edbd2adc'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:37.866608 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-926ff13c-6dc1-4049-a835-106ee4c24f5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69a83cba7cf94579998f9e24edbd2adc'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:37.866864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-717c1fc2-4774-4bd6-a17a-0ca1ffe0cb71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a73a00c3ce4496ebda509a7dde95b39'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:37.878613 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-717c1fc2-4774-4bd6-a17a-0ca1ffe0cb71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a73a00c3ce4496ebda509a7dde95b39'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:37.879598 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-717c1fc2-4774-4bd6-a17a-0ca1ffe0cb71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a73a00c3ce4496ebda509a7dde95b39'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:37.883787 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 970/1912] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:37 2022] DELETE /identity/v3/users/69a83cba7cf94579998f9e24edbd2adc => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:19:37.901687 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-717c1fc2-4774-4bd6-a17a-0ca1ffe0cb71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a73a00c3ce4496ebda509a7dde95b39'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:37.914451 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-717c1fc2-4774-4bd6-a17a-0ca1ffe0cb71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a73a00c3ce4496ebda509a7dde95b39'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:37.922581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 943/1913] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:37 2022] DELETE /identity/v3/users/0a73a00c3ce4496ebda509a7dde95b39 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:38.672345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ff13c-6dc1-4049-a835-106ee4c24f5b admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:38.674720 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-926ff13c-6dc1-4049-a835-106ee4c24f5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:38.683804 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c1fc2-4774-4bd6-a17a-0ca1ffe0cb71 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:38.686034 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-717c1fc2-4774-4bd6-a17a-0ca1ffe0cb71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:38.840555 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:38.897497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c1fc2-4774-4bd6-a17a-0ca1ffe0cb71 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:38.905520 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7954005-4bf7-41c1-a353-3ac2d5bc5b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:38.906249 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7954005-4bf7-41c1-a353-3ac2d5bc5b75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:38.906704 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7954005-4bf7-41c1-a353-3ac2d5bc5b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:38.907020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7954005-4bf7-41c1-a353-3ac2d5bc5b75 admin admin] PATH_INFO: `/v3/projects/b96b2c84f81d47afb301304a4ab486a0` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:38.909306 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7954005-4bf7-41c1-a353-3ac2d5bc5b75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b96b2c84f81d47afb301304a4ab486a0)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:38.911836 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7954005-4bf7-41c1-a353-3ac2d5bc5b75 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:38.940611 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ff13c-6dc1-4049-a835-106ee4c24f5b admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:38.946676 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15aee0a-aa42-47f2-817b-08412fbc5c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:38.948045 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15aee0a-aa42-47f2-817b-08412fbc5c68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:38.948045 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15aee0a-aa42-47f2-817b-08412fbc5c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:38.948045 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15aee0a-aa42-47f2-817b-08412fbc5c68 admin admin] PATH_INFO: `/v3/projects/79626f372a5044df8f88e3fd37a98d98` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:38.949291 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15aee0a-aa42-47f2-817b-08412fbc5c68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79626f372a5044df8f88e3fd37a98d98)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:38.952600 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15aee0a-aa42-47f2-817b-08412fbc5c68 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:39.012792 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-a7954005-4bf7-41c1-a353-3ac2d5bc5b75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b96b2c84f81d47afb301304a4ab486a0', 'request_id': 'req-a7954005-4bf7-41c1-a353-3ac2d5bc5b75'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:39.034558 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-a7954005-4bf7-41c1-a353-3ac2d5bc5b75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b96b2c84f81d47afb301304a4ab486a0', 'request_id': 'req-a7954005-4bf7-41c1-a353-3ac2d5bc5b75'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:39.046568 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-a7954005-4bf7-41c1-a353-3ac2d5bc5b75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b96b2c84f81d47afb301304a4ab486a0', 'request_id': 'req-a7954005-4bf7-41c1-a353-3ac2d5bc5b75'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:39.067073 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-a7954005-4bf7-41c1-a353-3ac2d5bc5b75 admin admin] The token cache is being invalidate because project b96b2c84f81d47afb301304a4ab486a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:39.067458 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-a7954005-4bf7-41c1-a353-3ac2d5bc5b75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:39.068143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-a7954005-4bf7-41c1-a353-3ac2d5bc5b75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:39.069129 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a15aee0a-aa42-47f2-817b-08412fbc5c68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79626f372a5044df8f88e3fd37a98d98', 'request_id': 'req-a15aee0a-aa42-47f2-817b-08412fbc5c68'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:39.070228 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 944/1914] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:38 2022] DELETE /identity/v3/projects/b96b2c84f81d47afb301304a4ab486a0 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:39.070714 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7954005-4bf7-41c1-a353-3ac2d5bc5b75 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:39.072599 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-a7954005-4bf7-41c1-a353-3ac2d5bc5b75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:39.083209 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a15aee0a-aa42-47f2-817b-08412fbc5c68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79626f372a5044df8f88e3fd37a98d98', 'request_id': 'req-a15aee0a-aa42-47f2-817b-08412fbc5c68'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:39.094431 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a15aee0a-aa42-47f2-817b-08412fbc5c68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79626f372a5044df8f88e3fd37a98d98', 'request_id': 'req-a15aee0a-aa42-47f2-817b-08412fbc5c68'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:39.117356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a15aee0a-aa42-47f2-817b-08412fbc5c68 admin admin] The token cache is being invalidate because project 79626f372a5044df8f88e3fd37a98d98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:39.117921 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a15aee0a-aa42-47f2-817b-08412fbc5c68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:39.120117 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a15aee0a-aa42-47f2-817b-08412fbc5c68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:39.122682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 971/1915] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:38 2022] DELETE /identity/v3/projects/79626f372a5044df8f88e3fd37a98d98 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:39.127873 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e8183-15d0-4d4f-8030-4d2995e58523 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:39.128165 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e8183-15d0-4d4f-8030-4d2995e58523 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:39.128448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e8183-15d0-4d4f-8030-4d2995e58523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:39.160290 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-8d0e8183-15d0-4d4f-8030-4d2995e58523 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:39.162239 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8d0e8183-15d0-4d4f-8030-4d2995e58523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:39.190775 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7954005-4bf7-41c1-a353-3ac2d5bc5b75 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:39.196531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:39.240293 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 972/1916] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:39.241891 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b5057-ffe7-4ad1-a265-d1d358e9f64f None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:39.242181 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b5057-ffe7-4ad1-a265-d1d358e9f64f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:39.242589 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b5057-ffe7-4ad1-a265-d1d358e9f64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:39.267715 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-855b5057-ffe7-4ad1-a265-d1d358e9f64f None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:39.271280 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-855b5057-ffe7-4ad1-a265-d1d358e9f64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:39.277996 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 973/1917] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:39.289309 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce21135-be5b-4945-93a9-93cc0ffe5e2b None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:39.289309 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce21135-be5b-4945-93a9-93cc0ffe5e2b None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:39.289309 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce21135-be5b-4945-93a9-93cc0ffe5e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:39.314877 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8fa75-7fa6-4448-aa3a-662a5bdeceb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:39.315920 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8fa75-7fa6-4448-aa3a-662a5bdeceb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:39.316203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8fa75-7fa6-4448-aa3a-662a5bdeceb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:39.316497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8fa75-7fa6-4448-aa3a-662a5bdeceb2 admin admin] PATH_INFO: `/v3/users/2c8370ecd5604849a9d9590ce536bb87` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:39.317811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d8fa75-7fa6-4448-aa3a-662a5bdeceb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c8370ecd5604849a9d9590ce536bb87)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:39.321710 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d8fa75-7fa6-4448-aa3a-662a5bdeceb2 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:39.354409 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-1ce21135-be5b-4945-93a9-93cc0ffe5e2b None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:39.357626 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1ce21135-be5b-4945-93a9-93cc0ffe5e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:39.364688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:39.400554 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 974/1918] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:39.400554 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b4d76-8f34-44fe-a0dd-1ab240e9fbe0 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:39.400554 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b4d76-8f34-44fe-a0dd-1ab240e9fbe0 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:39.400554 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b4d76-8f34-44fe-a0dd-1ab240e9fbe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:39.440175 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-f65b4d76-8f34-44fe-a0dd-1ab240e9fbe0 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:39.444321 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f65b4d76-8f34-44fe-a0dd-1ab240e9fbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:39.468134 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 975/1919] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:39.469206 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b4d76-8f34-44fe-a0dd-1ab240e9fbe0 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:39.478008 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-00d8fa75-7fa6-4448-aa3a-662a5bdeceb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c8370ecd5604849a9d9590ce536bb87'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:39.500202 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-00d8fa75-7fa6-4448-aa3a-662a5bdeceb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c8370ecd5604849a9d9590ce536bb87'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:39.503293 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-00d8fa75-7fa6-4448-aa3a-662a5bdeceb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c8370ecd5604849a9d9590ce536bb87'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:39.517761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b4d76-8f34-44fe-a0dd-1ab240e9fbe0 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:39.529728 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6502d2e-dfaa-424d-8054-503e82822dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:39.530431 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6502d2e-dfaa-424d-8054-503e82822dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:39.530636 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6502d2e-dfaa-424d-8054-503e82822dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:39.531841 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6502d2e-dfaa-424d-8054-503e82822dc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:39.539718 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-00d8fa75-7fa6-4448-aa3a-662a5bdeceb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c8370ecd5604849a9d9590ce536bb87'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:39.549706 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6502d2e-dfaa-424d-8054-503e82822dc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:19:39.550255 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6502d2e-dfaa-424d-8054-503e82822dc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:39.553966 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6502d2e-dfaa-424d-8054-503e82822dc9 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:39.560321 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-00d8fa75-7fa6-4448-aa3a-662a5bdeceb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c8370ecd5604849a9d9590ce536bb87'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:39.575680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 945/1920] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:39 2022] DELETE /identity/v3/users/2c8370ecd5604849a9d9590ce536bb87 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:39.576724 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8fa75-7fa6-4448-aa3a-662a5bdeceb2 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:39.578384 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-00d8fa75-7fa6-4448-aa3a-662a5bdeceb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:39.590102 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 976/1921] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:19:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:19:39.593785 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be1c4c-3de9-42ef-8d6d-8423e578a7a6 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:39.596396 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be1c4c-3de9-42ef-8d6d-8423e578a7a6 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:39.599074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be1c4c-3de9-42ef-8d6d-8423e578a7a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:39.642401 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-66be1c4c-3de9-42ef-8d6d-8423e578a7a6 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:39.644206 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-66be1c4c-3de9-42ef-8d6d-8423e578a7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:39.701956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8fa75-7fa6-4448-aa3a-662a5bdeceb2 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:39.710248 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc21c12-9229-46a6-82ee-f5964fef754f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:39.711414 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc21c12-9229-46a6-82ee-f5964fef754f admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:39.712440 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc21c12-9229-46a6-82ee-f5964fef754f admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:39.713843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc21c12-9229-46a6-82ee-f5964fef754f admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:39.714772 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc21c12-9229-46a6-82ee-f5964fef754f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:19:39.715086 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc21c12-9229-46a6-82ee-f5964fef754f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:39.717990 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc21c12-9229-46a6-82ee-f5964fef754f admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:39.741828 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 946/1922] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:19:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:19:39.741828 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc21c12-9229-46a6-82ee-f5964fef754f admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:39.744686 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 977/1923] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:39.745971 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be1c4c-3de9-42ef-8d6d-8423e578a7a6 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:39.747026 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-3cc21c12-9229-46a6-82ee-f5964fef754f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:39.763141 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be1c4c-3de9-42ef-8d6d-8423e578a7a6 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:39.770537 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edbbfd7-1247-46d9-ad19-8cb9febd93d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:39.771372 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbbfd7-1247-46d9-ad19-8cb9febd93d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:39.771778 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbbfd7-1247-46d9-ad19-8cb9febd93d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:39.772153 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbbfd7-1247-46d9-ad19-8cb9febd93d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:39.773116 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edbbfd7-1247-46d9-ad19-8cb9febd93d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:39.773669 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc21c12-9229-46a6-82ee-f5964fef754f admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:39.775625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edbbfd7-1247-46d9-ad19-8cb9febd93d1 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:39.784185 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a86d9d-8a81-45c7-93e0-e4be0f4adfd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:39.785507 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a86d9d-8a81-45c7-93e0-e4be0f4adfd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:39.786204 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a86d9d-8a81-45c7-93e0-e4be0f4adfd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:39.786861 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a86d9d-8a81-45c7-93e0-e4be0f4adfd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:39.788336 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a86d9d-8a81-45c7-93e0-e4be0f4adfd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:39.792434 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a86d9d-8a81-45c7-93e0-e4be0f4adfd7 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:39.823576 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 978/1924] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:39 2022] POST /identity/v3/projects => generated 371 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:39.823576 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edbbfd7-1247-46d9-ad19-8cb9febd93d1 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:39.827436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-0edbbfd7-1247-46d9-ad19-8cb9febd93d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:39.854638 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 947/1925] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:39 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:39.855604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a86d9d-8a81-45c7-93e0-e4be0f4adfd7 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:39.913890 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edbbfd7-1247-46d9-ad19-8cb9febd93d1 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:39.960509 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a86d9d-8a81-45c7-93e0-e4be0f4adfd7 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:39.977522 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46da3df-5d83-45de-a5e6-48ab0db1c932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:39.978544 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:39.978719 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46da3df-5d83-45de-a5e6-48ab0db1c932 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:39.978927 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46da3df-5d83-45de-a5e6-48ab0db1c932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:39.980096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46da3df-5d83-45de-a5e6-48ab0db1c932 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:39.980810 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46da3df-5d83-45de-a5e6-48ab0db1c932 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:39.983282 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46da3df-5d83-45de-a5e6-48ab0db1c932 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:39.997162 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:40.021106 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 948/1926] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:39 2022] POST /identity/v3/users => generated 286 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:40.022181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46da3df-5d83-45de-a5e6-48ab0db1c932 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:40.027535 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210b5c8-9ff7-4d8f-8354-bfd7169f7ceb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:40.032676 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210b5c8-9ff7-4d8f-8354-bfd7169f7ceb service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:40.033965 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210b5c8-9ff7-4d8f-8354-bfd7169f7ceb service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:40.039735 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210b5c8-9ff7-4d8f-8354-bfd7169f7ceb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:40.040634 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46da3df-5d83-45de-a5e6-48ab0db1c932 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:40.067963 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b210b5c8-9ff7-4d8f-8354-bfd7169f7ceb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:40.072033 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b210b5c8-9ff7-4d8f-8354-bfd7169f7ceb service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:40.080373 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7c7b4-d034-4288-91fa-5387363fd3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:40.081131 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7c7b4-d034-4288-91fa-5387363fd3e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:40.081418 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7c7b4-d034-4288-91fa-5387363fd3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:40.081690 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7c7b4-d034-4288-91fa-5387363fd3e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:40.082346 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e7c7b4-d034-4288-91fa-5387363fd3e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:40.086423 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 979/1927] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:40.088449 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210b5c8-9ff7-4d8f-8354-bfd7169f7ceb service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:40.100479 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210b5c8-9ff7-4d8f-8354-bfd7169f7ceb service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:40.104173 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e7c7b4-d034-4288-91fa-5387363fd3e0 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:40.107411 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d6897-9800-4b0a-a654-051bc53b4523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:40.109156 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d6897-9800-4b0a-a654-051bc53b4523 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:40.109532 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d6897-9800-4b0a-a654-051bc53b4523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:40.109882 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d6897-9800-4b0a-a654-051bc53b4523 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:40.112958 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786d6897-9800-4b0a-a654-051bc53b4523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:40.114756 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786d6897-9800-4b0a-a654-051bc53b4523 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:40.140168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 980/1928] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:40.146155 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d6897-9800-4b0a-a654-051bc53b4523 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:40.163963 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d6897-9800-4b0a-a654-051bc53b4523 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:40.171049 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c0b30-adda-48b3-b073-9fde8c4b8495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:40.171049 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c0b30-adda-48b3-b073-9fde8c4b8495 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:40.171446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c0b30-adda-48b3-b073-9fde8c4b8495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:40.171762 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c0b30-adda-48b3-b073-9fde8c4b8495 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:40.172739 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c0b30-adda-48b3-b073-9fde8c4b8495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:40.174490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c0b30-adda-48b3-b073-9fde8c4b8495 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:40.188636 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 981/1929] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:40.197795 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 949/1930] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:40 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:40.202776 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c0b30-adda-48b3-b073-9fde8c4b8495 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:40.203764 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7c7b4-d034-4288-91fa-5387363fd3e0 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:40.217636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7c7b4-d034-4288-91fa-5387363fd3e0 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:40.226550 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38d803-6353-4087-8f5c-f56d631a2f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:40.227480 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38d803-6353-4087-8f5c-f56d631a2f60 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:40.227864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38d803-6353-4087-8f5c-f56d631a2f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:40.228358 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38d803-6353-4087-8f5c-f56d631a2f60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:40.229019 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c0b30-adda-48b3-b073-9fde8c4b8495 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:40.229369 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff38d803-6353-4087-8f5c-f56d631a2f60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:40.232390 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff38d803-6353-4087-8f5c-f56d631a2f60 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:40.253516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61e400-274b-4fac-a6d6-cd281eee28c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:40.255807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 950/1931] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:40.256977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61e400-274b-4fac-a6d6-cd281eee28c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:40.257477 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61e400-274b-4fac-a6d6-cd281eee28c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:40.258606 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61e400-274b-4fac-a6d6-cd281eee28c2 admin admin] PATH_INFO: `/v3/projects/707b77c7fc104ae0853e057724a6d21e/users/8ba4126b54a04efdb717aa174a65368c/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:40.267703 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38d803-6353-4087-8f5c-f56d631a2f60 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:40.283251 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38d803-6353-4087-8f5c-f56d631a2f60 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:40.293128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93474737-5e58-49fa-a5d3-ab2a447e4be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:40.294447 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93474737-5e58-49fa-a5d3-ab2a447e4be2 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:40.295222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93474737-5e58-49fa-a5d3-ab2a447e4be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:40.295875 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93474737-5e58-49fa-a5d3-ab2a447e4be2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:40.297314 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93474737-5e58-49fa-a5d3-ab2a447e4be2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:40.299835 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93474737-5e58-49fa-a5d3-ab2a447e4be2 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:40.307999 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f61e400-274b-4fac-a6d6-cd281eee28c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=707b77c7fc104ae0853e057724a6d21e, user_id=8ba4126b54a04efdb717aa174a65368c, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:40.310744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f61e400-274b-4fac-a6d6-cd281eee28c2 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:40.323895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 951/1932] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:40.332868 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93474737-5e58-49fa-a5d3-ab2a447e4be2 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:40.351952 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 982/1933] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:40 2022] PUT /identity/v3/projects/707b77c7fc104ae0853e057724a6d21e/users/8ba4126b54a04efdb717aa174a65368c/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:19:40.359535 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c6b1f-9ae4-4b8d-90b4-8f77805ef0f3 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:40.359830 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c6b1f-9ae4-4b8d-90b4-8f77805ef0f3 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:40.360103 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c6b1f-9ae4-4b8d-90b4-8f77805ef0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:40.374231 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93474737-5e58-49fa-a5d3-ab2a447e4be2 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:40.440783 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-163c6b1f-9ae4-4b8d-90b4-8f77805ef0f3 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:40.441957 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-163c6b1f-9ae4-4b8d-90b4-8f77805ef0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:40.476817 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66ca3f-772f-46d6-b42a-976707bb0c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:40.477625 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66ca3f-772f-46d6-b42a-976707bb0c61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:40.478513 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66ca3f-772f-46d6-b42a-976707bb0c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:40.484179 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66ca3f-772f-46d6-b42a-976707bb0c61 admin admin] PATH_INFO: `/v3/projects/5523d6c106da44dd8dec059df60f2f18/users/5510817494fe4f2cb546a0102c364f1f/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:40.517310 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c66ca3f-772f-46d6-b42a-976707bb0c61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5523d6c106da44dd8dec059df60f2f18, user_id=5510817494fe4f2cb546a0102c364f1f, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:40.520496 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c66ca3f-772f-46d6-b42a-976707bb0c61 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:40.551285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 952/1934] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:40 2022] PUT /identity/v3/projects/5523d6c106da44dd8dec059df60f2f18/users/5510817494fe4f2cb546a0102c364f1f/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:40.552738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4192b-3de8-40d7-8475-a9b21f740e58 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:40.553027 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4192b-3de8-40d7-8475-a9b21f740e58 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:40.553382 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4192b-3de8-40d7-8475-a9b21f740e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:40.633832 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 983/1935] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:19:40.646602 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-cff4192b-3de8-40d7-8475-a9b21f740e58 None None] MFA Rules not processed for user `5510817494fe4f2cb546a0102c364f1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:40.648751 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-cff4192b-3de8-40d7-8475-a9b21f740e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:40.652465 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c6b1f-9ae4-4b8d-90b4-8f77805ef0f3 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:40.754414 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c6b1f-9ae4-4b8d-90b4-8f77805ef0f3 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:40.769305 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 953/1936] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:40.776178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4192b-3de8-40d7-8475-a9b21f740e58 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:40.814152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:40.829750 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:40.833672 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed930a-ca2b-4465-b642-07e687ec3073 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:40.834810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed930a-ca2b-4465-b642-07e687ec3073 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:40.835224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed930a-ca2b-4465-b642-07e687ec3073 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:40.836382 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed930a-ca2b-4465-b642-07e687ec3073 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:40.901291 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4192b-3de8-40d7-8475-a9b21f740e58 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:40.909111 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce52b9f-3b67-496a-88e5-2f86843bbe75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:40.909896 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce52b9f-3b67-496a-88e5-2f86843bbe75 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:40.910315 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce52b9f-3b67-496a-88e5-2f86843bbe75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:40.910718 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce52b9f-3b67-496a-88e5-2f86843bbe75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:40.911492 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce52b9f-3b67-496a-88e5-2f86843bbe75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:40.913935 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce52b9f-3b67-496a-88e5-2f86843bbe75 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:40.954634 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed930a-ca2b-4465-b642-07e687ec3073 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:40.954838 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 954/1937] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:40 2022] POST /identity/v3/projects => generated 385 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:40.955606 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce52b9f-3b67-496a-88e5-2f86843bbe75 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:40.956932 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed930a-ca2b-4465-b642-07e687ec3073 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:40.957277 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-4ce52b9f-3b67-496a-88e5-2f86843bbe75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:40.974348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 984/1938] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:19:40.991164 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed930a-ca2b-4465-b642-07e687ec3073 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:41.010044 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed930a-ca2b-4465-b642-07e687ec3073 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:41.102321 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce52b9f-3b67-496a-88e5-2f86843bbe75 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:41.113153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a351e9-b8ae-48b7-9ab8-196e69ab86c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:41.114524 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a351e9-b8ae-48b7-9ab8-196e69ab86c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:41.115065 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a351e9-b8ae-48b7-9ab8-196e69ab86c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:41.115549 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a351e9-b8ae-48b7-9ab8-196e69ab86c6 admin admin] PATH_INFO: `/v3/projects/cfaa5c42b3844316a71fb08d24735f04` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:41.117194 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a351e9-b8ae-48b7-9ab8-196e69ab86c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfaa5c42b3844316a71fb08d24735f04)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:41.118149 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12f528-d484-4df4-99e2-7d0301b44c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:41.119136 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12f528-d484-4df4-99e2-7d0301b44c7a admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:41.119136 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12f528-d484-4df4-99e2-7d0301b44c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:41.119401 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12f528-d484-4df4-99e2-7d0301b44c7a admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:41.120966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a351e9-b8ae-48b7-9ab8-196e69ab86c6 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:41.121233 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12f528-d484-4df4-99e2-7d0301b44c7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:41.123962 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12f528-d484-4df4-99e2-7d0301b44c7a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:41.179159 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 985/1939] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:40 2022] POST /identity/v3/users => generated 299 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:41.183270 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12f528-d484-4df4-99e2-7d0301b44c7a admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:41.205326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12f528-d484-4df4-99e2-7d0301b44c7a admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:41.214467 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-67a351e9-b8ae-48b7-9ab8-196e69ab86c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfaa5c42b3844316a71fb08d24735f04', 'request_id': 'req-67a351e9-b8ae-48b7-9ab8-196e69ab86c6'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:41.229816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-67a351e9-b8ae-48b7-9ab8-196e69ab86c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfaa5c42b3844316a71fb08d24735f04', 'request_id': 'req-67a351e9-b8ae-48b7-9ab8-196e69ab86c6'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:41.241317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-67a351e9-b8ae-48b7-9ab8-196e69ab86c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfaa5c42b3844316a71fb08d24735f04', 'request_id': 'req-67a351e9-b8ae-48b7-9ab8-196e69ab86c6'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:41.264800 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-67a351e9-b8ae-48b7-9ab8-196e69ab86c6 admin admin] The token cache is being invalidate because project cfaa5c42b3844316a71fb08d24735f04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:41.265446 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-67a351e9-b8ae-48b7-9ab8-196e69ab86c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:41.266565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-67a351e9-b8ae-48b7-9ab8-196e69ab86c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:41.272295 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 955/1940] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:40 2022] DELETE /identity/v3/projects/cfaa5c42b3844316a71fb08d24735f04 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:41.278833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a351e9-b8ae-48b7-9ab8-196e69ab86c6 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:41.281661 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-67a351e9-b8ae-48b7-9ab8-196e69ab86c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:41.340307 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6c919-6db6-4872-a678-e7bced046534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:41.341305 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6c919-6db6-4872-a678-e7bced046534 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:41.342455 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6c919-6db6-4872-a678-e7bced046534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:41.342900 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6c919-6db6-4872-a678-e7bced046534 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:41.345548 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6c919-6db6-4872-a678-e7bced046534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:41.348612 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6c919-6db6-4872-a678-e7bced046534 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:41.385969 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:41.393324 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 986/1941] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:19:41.399004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6c919-6db6-4872-a678-e7bced046534 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:41.404950 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5de6c919-6db6-4872-a678-e7bced046534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:41.454484 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6c919-6db6-4872-a678-e7bced046534 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:41.456639 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a351e9-b8ae-48b7-9ab8-196e69ab86c6 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:41.465585 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac639a-cf5a-4485-8001-bbae9a9fcd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:41.467173 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac639a-cf5a-4485-8001-bbae9a9fcd0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:41.467790 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac639a-cf5a-4485-8001-bbae9a9fcd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:41.468471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac639a-cf5a-4485-8001-bbae9a9fcd0e admin admin] PATH_INFO: `/v3/projects/6fdf284bc2504b20a28bd9b21d2564a4/users/de1b2d01d632470a9aadbe0fb2a30e19/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:41.473552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ddf73-0e9f-4521-9a6e-a2d7d5be50d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:41.474297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ddf73-0e9f-4521-9a6e-a2d7d5be50d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:41.474634 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ddf73-0e9f-4521-9a6e-a2d7d5be50d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:41.474844 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ddf73-0e9f-4521-9a6e-a2d7d5be50d7 admin admin] PATH_INFO: `/v3/users/ab485ef5613e447f894358551986fd1a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:41.482839 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3ddf73-0e9f-4521-9a6e-a2d7d5be50d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab485ef5613e447f894358551986fd1a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:41.488736 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3ddf73-0e9f-4521-9a6e-a2d7d5be50d7 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:41.501282 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac639a-cf5a-4485-8001-bbae9a9fcd0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fdf284bc2504b20a28bd9b21d2564a4, user_id=de1b2d01d632470a9aadbe0fb2a30e19, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:41.507207 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac639a-cf5a-4485-8001-bbae9a9fcd0e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:41.513175 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:41.539829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 987/1942] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:41 2022] PUT /identity/v3/projects/6fdf284bc2504b20a28bd9b21d2564a4/users/de1b2d01d632470a9aadbe0fb2a30e19/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:41.546808 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac639a-cf5a-4485-8001-bbae9a9fcd0e admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:41.631645 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-ac3ddf73-0e9f-4521-9a6e-a2d7d5be50d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab485ef5613e447f894358551986fd1a'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:41.681470 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-ac3ddf73-0e9f-4521-9a6e-a2d7d5be50d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab485ef5613e447f894358551986fd1a'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:41.685358 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-ac3ddf73-0e9f-4521-9a6e-a2d7d5be50d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab485ef5613e447f894358551986fd1a'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:41.732232 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-ac3ddf73-0e9f-4521-9a6e-a2d7d5be50d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab485ef5613e447f894358551986fd1a'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:41.765035 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-ac3ddf73-0e9f-4521-9a6e-a2d7d5be50d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab485ef5613e447f894358551986fd1a'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:41.785211 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac639a-cf5a-4485-8001-bbae9a9fcd0e admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:41.798571 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780f7aa-e014-469c-b741-99d4e5c2d883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:41.800483 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780f7aa-e014-469c-b741-99d4e5c2d883 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:41.803185 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780f7aa-e014-469c-b741-99d4e5c2d883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:41.804149 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780f7aa-e014-469c-b741-99d4e5c2d883 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:41.805584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780f7aa-e014-469c-b741-99d4e5c2d883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:41.809660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780f7aa-e014-469c-b741-99d4e5c2d883 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:41.819815 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 956/1943] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:41 2022] DELETE /identity/v3/users/ab485ef5613e447f894358551986fd1a => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:41.821075 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ddf73-0e9f-4521-9a6e-a2d7d5be50d7 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:41.833613 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-ac3ddf73-0e9f-4521-9a6e-a2d7d5be50d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:41.844256 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 988/1944] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:41.850615 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780f7aa-e014-469c-b741-99d4e5c2d883 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:41.855919 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b780f7aa-e014-469c-b741-99d4e5c2d883 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:41.889861 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780f7aa-e014-469c-b741-99d4e5c2d883 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:42.093249 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a46df2-baf4-47e3-bca7-8c6e1a1f1436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:42.098332 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a46df2-baf4-47e3-bca7-8c6e1a1f1436 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:42.098834 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a46df2-baf4-47e3-bca7-8c6e1a1f1436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:42.099421 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a46df2-baf4-47e3-bca7-8c6e1a1f1436 admin admin] PATH_INFO: `/v3/projects/6fdf284bc2504b20a28bd9b21d2564a4/users/de1b2d01d632470a9aadbe0fb2a30e19/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:42.112831 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a46df2-baf4-47e3-bca7-8c6e1a1f1436 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fdf284bc2504b20a28bd9b21d2564a4, user_id=de1b2d01d632470a9aadbe0fb2a30e19, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:42.124144 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a46df2-baf4-47e3-bca7-8c6e1a1f1436 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:42.131366 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ddf73-0e9f-4521-9a6e-a2d7d5be50d7 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:42.160841 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddf60b8-2bdb-4e97-bb09-924f7c089cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:42.161622 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf60b8-2bdb-4e97-bb09-924f7c089cf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:42.161934 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf60b8-2bdb-4e97-bb09-924f7c089cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:42.162611 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf60b8-2bdb-4e97-bb09-924f7c089cf4 admin admin] PATH_INFO: `/v3/users/d4c723489b5643a9a60a34c513b265c0` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:42.165206 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddf60b8-2bdb-4e97-bb09-924f7c089cf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4c723489b5643a9a60a34c513b265c0)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:42.182484 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddf60b8-2bdb-4e97-bb09-924f7c089cf4 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:42.227603 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 989/1945] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:41 2022] PUT /identity/v3/projects/6fdf284bc2504b20a28bd9b21d2564a4/users/de1b2d01d632470a9aadbe0fb2a30e19/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:42.240963 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b01648-9986-4e1f-948e-0da9adbe13ff None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:42.241386 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b01648-9986-4e1f-948e-0da9adbe13ff None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:42.241595 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b01648-9986-4e1f-948e-0da9adbe13ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:42.326768 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-47b01648-9986-4e1f-948e-0da9adbe13ff None None] MFA Rules not processed for user `de1b2d01d632470a9aadbe0fb2a30e19`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:42.328698 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-47b01648-9986-4e1f-948e-0da9adbe13ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:42.350331 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-bddf60b8-2bdb-4e97-bb09-924f7c089cf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4c723489b5643a9a60a34c513b265c0'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:42.379776 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-bddf60b8-2bdb-4e97-bb09-924f7c089cf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4c723489b5643a9a60a34c513b265c0'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:42.380409 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-bddf60b8-2bdb-4e97-bb09-924f7c089cf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4c723489b5643a9a60a34c513b265c0'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:42.401115 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:42.417960 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-bddf60b8-2bdb-4e97-bb09-924f7c089cf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4c723489b5643a9a60a34c513b265c0'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:42.450216 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-bddf60b8-2bdb-4e97-bb09-924f7c089cf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4c723489b5643a9a60a34c513b265c0'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:42.479794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 957/1946] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:41 2022] DELETE /identity/v3/users/d4c723489b5643a9a60a34c513b265c0 => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:42.480899 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fb6ac-0a42-4d2c-897f-fae2ef4acaaa None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:42.481195 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fb6ac-0a42-4d2c-897f-fae2ef4acaaa None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:42.481476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fb6ac-0a42-4d2c-897f-fae2ef4acaaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:42.531378 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 990/1947] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:42 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:42.541144 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-074fb6ac-0a42-4d2c-897f-fae2ef4acaaa None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:42.542603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-074fb6ac-0a42-4d2c-897f-fae2ef4acaaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:42.547400 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b01648-9986-4e1f-948e-0da9adbe13ff None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:42.549549 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-47b01648-9986-4e1f-948e-0da9adbe13ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:42.584500 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:42.700992 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 958/1948] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:42.701211 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b01648-9986-4e1f-948e-0da9adbe13ff None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:42.739372 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074fb6ac-0a42-4d2c-897f-fae2ef4acaaa None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:42.741397 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-074fb6ac-0a42-4d2c-897f-fae2ef4acaaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:42.776194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed80ebe-54f6-4986-abf1-d1edcc7037f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:42.778044 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed80ebe-54f6-4986-abf1-d1edcc7037f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:42.778695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed80ebe-54f6-4986-abf1-d1edcc7037f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:42.779794 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed80ebe-54f6-4986-abf1-d1edcc7037f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:42.783797 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8ed80ebe-54f6-4986-abf1-d1edcc7037f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:42.842694 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074fb6ac-0a42-4d2c-897f-fae2ef4acaaa None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:42.849576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc21dc-ea78-4bee-8f13-17e4954e7f92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:42.850684 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc21dc-ea78-4bee-8f13-17e4954e7f92 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:42.851121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc21dc-ea78-4bee-8f13-17e4954e7f92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:42.851663 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc21dc-ea78-4bee-8f13-17e4954e7f92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:42.870655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc21dc-ea78-4bee-8f13-17e4954e7f92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:42.872188 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc21dc-ea78-4bee-8f13-17e4954e7f92 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:42.910795 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 959/1949] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:42.918299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed80ebe-54f6-4986-abf1-d1edcc7037f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:42.926130 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed80ebe-54f6-4986-abf1-d1edcc7037f4 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:42.944795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 991/1950] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:19:43.193048 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc21dc-ea78-4bee-8f13-17e4954e7f92 service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:43.195153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-bdfc21dc-ea78-4bee-8f13-17e4954e7f92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:43.213038 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc21dc-ea78-4bee-8f13-17e4954e7f92 service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:43.247155 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856758aa-46a4-47d3-97aa-1c670badcd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:43.248891 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856758aa-46a4-47d3-97aa-1c670badcd4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:43.250421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856758aa-46a4-47d3-97aa-1c670badcd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:43.251772 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856758aa-46a4-47d3-97aa-1c670badcd4f admin admin] PATH_INFO: `/v3/projects/8721adad308f486892506dc546d55bad` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:43.255932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856758aa-46a4-47d3-97aa-1c670badcd4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8721adad308f486892506dc546d55bad)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:43.262477 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856758aa-46a4-47d3-97aa-1c670badcd4f admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:43.343785 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed80ebe-54f6-4986-abf1-d1edcc7037f4 service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:43.346920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8ed80ebe-54f6-4986-abf1-d1edcc7037f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:43.377721 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-856758aa-46a4-47d3-97aa-1c670badcd4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8721adad308f486892506dc546d55bad', 'request_id': 'req-856758aa-46a4-47d3-97aa-1c670badcd4f'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:43.378636 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed80ebe-54f6-4986-abf1-d1edcc7037f4 service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:43.389906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-856758aa-46a4-47d3-97aa-1c670badcd4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8721adad308f486892506dc546d55bad', 'request_id': 'req-856758aa-46a4-47d3-97aa-1c670badcd4f'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:43.400239 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-856758aa-46a4-47d3-97aa-1c670badcd4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8721adad308f486892506dc546d55bad', 'request_id': 'req-856758aa-46a4-47d3-97aa-1c670badcd4f'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:43.420412 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-856758aa-46a4-47d3-97aa-1c670badcd4f admin admin] The token cache is being invalidate because project 8721adad308f486892506dc546d55bad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:43.420886 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-856758aa-46a4-47d3-97aa-1c670badcd4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:43.422510 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-856758aa-46a4-47d3-97aa-1c670badcd4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:43.424801 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 960/1951] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:43 2022] DELETE /identity/v3/projects/8721adad308f486892506dc546d55bad => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:43.427031 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a23680-44a2-48e3-b047-be244db9ae86 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:43.427501 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a23680-44a2-48e3-b047-be244db9ae86 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:43.428064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a23680-44a2-48e3-b047-be244db9ae86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:43.450877 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:43.475670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-51a23680-44a2-48e3-b047-be244db9ae86 None None] MFA Rules not processed for user `8ba4126b54a04efdb717aa174a65368c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:43.476942 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-51a23680-44a2-48e3-b047-be244db9ae86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:43.547025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880a842-c512-40fd-8c16-17b67c560a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:43.548526 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880a842-c512-40fd-8c16-17b67c560a92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:43.548702 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880a842-c512-40fd-8c16-17b67c560a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:43.548995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880a842-c512-40fd-8c16-17b67c560a92 admin admin] PATH_INFO: `/v3/projects/849b392010134a59aaa39da54fc15152` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:43.550995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9880a842-c512-40fd-8c16-17b67c560a92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=849b392010134a59aaa39da54fc15152)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:43.555828 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9880a842-c512-40fd-8c16-17b67c560a92 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:43.614046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:43.667557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-9880a842-c512-40fd-8c16-17b67c560a92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '849b392010134a59aaa39da54fc15152', 'request_id': 'req-9880a842-c512-40fd-8c16-17b67c560a92'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:43.682599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-9880a842-c512-40fd-8c16-17b67c560a92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '849b392010134a59aaa39da54fc15152', 'request_id': 'req-9880a842-c512-40fd-8c16-17b67c560a92'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:43.698392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 961/1952] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:43.698825 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-9880a842-c512-40fd-8c16-17b67c560a92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '849b392010134a59aaa39da54fc15152', 'request_id': 'req-9880a842-c512-40fd-8c16-17b67c560a92'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:43.698911 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4ff46-56df-4f1f-b3ed-9e8c11ba9ff0 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:43.699369 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4ff46-56df-4f1f-b3ed-9e8c11ba9ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:43.700596 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4ff46-56df-4f1f-b3ed-9e8c11ba9ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:43.724680 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-9880a842-c512-40fd-8c16-17b67c560a92 admin admin] The token cache is being invalidate because project 849b392010134a59aaa39da54fc15152 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:43.724970 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-9880a842-c512-40fd-8c16-17b67c560a92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:43.729829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-9880a842-c512-40fd-8c16-17b67c560a92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:43.733012 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 992/1953] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:43 2022] DELETE /identity/v3/projects/849b392010134a59aaa39da54fc15152 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:43.738593 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880a842-c512-40fd-8c16-17b67c560a92 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:43.742583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9880a842-c512-40fd-8c16-17b67c560a92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:43.755964 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-5df4ff46-56df-4f1f-b3ed-9e8c11ba9ff0 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:43.759801 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-5df4ff46-56df-4f1f-b3ed-9e8c11ba9ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:43.862808 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880a842-c512-40fd-8c16-17b67c560a92 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:43.868105 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa8728f-1c98-4d3b-8994-f60a5a0a532a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:43.868736 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8728f-1c98-4d3b-8994-f60a5a0a532a admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:43.869016 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8728f-1c98-4d3b-8994-f60a5a0a532a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:43.869290 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8728f-1c98-4d3b-8994-f60a5a0a532a admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:43.869937 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa8728f-1c98-4d3b-8994-f60a5a0a532a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:43.871918 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa8728f-1c98-4d3b-8994-f60a5a0a532a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:43.880874 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 962/1954] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:43.889090 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4ff46-56df-4f1f-b3ed-9e8c11ba9ff0 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:43.890602 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-5df4ff46-56df-4f1f-b3ed-9e8c11ba9ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:43.908171 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4ff46-56df-4f1f-b3ed-9e8c11ba9ff0 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:43.912414 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 993/1955] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:43 2022] POST /identity/v3/projects => generated 373 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:43.913918 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00b818-311e-4f1e-8d7c-fe17b87154b7 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:43.914207 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00b818-311e-4f1e-8d7c-fe17b87154b7 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:43.914488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00b818-311e-4f1e-8d7c-fe17b87154b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:43.916979 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737db027-7459-4bc9-9118-75fe656f3998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:43.917769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737db027-7459-4bc9-9118-75fe656f3998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:43.918114 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737db027-7459-4bc9-9118-75fe656f3998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:43.918458 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737db027-7459-4bc9-9118-75fe656f3998 admin admin] PATH_INFO: `/v3/users/7b89bcae79854d869c8bf39c23c8f700` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:43.919741 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737db027-7459-4bc9-9118-75fe656f3998 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b89bcae79854d869c8bf39c23c8f700)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:43.921859 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737db027-7459-4bc9-9118-75fe656f3998 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:43.945448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-7d00b818-311e-4f1e-8d7c-fe17b87154b7 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:43.948093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7d00b818-311e-4f1e-8d7c-fe17b87154b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:44.016262 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-737db027-7459-4bc9-9118-75fe656f3998 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b89bcae79854d869c8bf39c23c8f700'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:44.040326 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-737db027-7459-4bc9-9118-75fe656f3998 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b89bcae79854d869c8bf39c23c8f700'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:44.044455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-737db027-7459-4bc9-9118-75fe656f3998 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b89bcae79854d869c8bf39c23c8f700'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:44.066877 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-737db027-7459-4bc9-9118-75fe656f3998 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b89bcae79854d869c8bf39c23c8f700'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:44.075676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-737db027-7459-4bc9-9118-75fe656f3998 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b89bcae79854d869c8bf39c23c8f700'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:44.087051 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 963/1956] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:43 2022] DELETE /identity/v3/users/7b89bcae79854d869c8bf39c23c8f700 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:44.087051 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737db027-7459-4bc9-9118-75fe656f3998 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:44.088698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-737db027-7459-4bc9-9118-75fe656f3998 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:44.102555 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:44.110656 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 994/1957] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:44.117270 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00b818-311e-4f1e-8d7c-fe17b87154b7 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:44.135860 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00b818-311e-4f1e-8d7c-fe17b87154b7 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:44.215327 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737db027-7459-4bc9-9118-75fe656f3998 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:44.226548 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2086af-abb4-4032-b962-7eac24395c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:44.227169 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:44.227321 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2086af-abb4-4032-b962-7eac24395c2d admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:44.227432 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2086af-abb4-4032-b962-7eac24395c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:44.228737 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2086af-abb4-4032-b962-7eac24395c2d admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:44.231724 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2086af-abb4-4032-b962-7eac24395c2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:44.236080 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2086af-abb4-4032-b962-7eac24395c2d admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:44.278606 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 964/1958] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:44 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:44.281165 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14f615-5328-44f2-8444-f858adcbbad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:44.282912 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14f615-5328-44f2-8444-f858adcbbad1 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:44.283349 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14f615-5328-44f2-8444-f858adcbbad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:44.283772 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14f615-5328-44f2-8444-f858adcbbad1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:44.284001 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2086af-abb4-4032-b962-7eac24395c2d admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:44.285449 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14f615-5328-44f2-8444-f858adcbbad1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:19:44.286029 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14f615-5328-44f2-8444-f858adcbbad1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:44.296551 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14f615-5328-44f2-8444-f858adcbbad1 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:44.302193 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2086af-abb4-4032-b962-7eac24395c2d admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:44.324674 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea91e1-bc4b-4464-8188-80e3e1416fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:44.325204 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea91e1-bc4b-4464-8188-80e3e1416fd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:44.325491 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea91e1-bc4b-4464-8188-80e3e1416fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:44.325775 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea91e1-bc4b-4464-8188-80e3e1416fd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:44.326447 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ea91e1-bc4b-4464-8188-80e3e1416fd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:44.328162 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ea91e1-bc4b-4464-8188-80e3e1416fd0 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:44.338506 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 995/1959] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:19:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:19:44.346048 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14f615-5328-44f2-8444-f858adcbbad1 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:44.351577 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 965/1960] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:44.357483 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea91e1-bc4b-4464-8188-80e3e1416fd0 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:44.374202 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14f615-5328-44f2-8444-f858adcbbad1 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:44.383085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea91e1-bc4b-4464-8188-80e3e1416fd0 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:44.391023 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc3870-db48-47de-a582-998f586580fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:44.392157 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc3870-db48-47de-a582-998f586580fd admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:44.392933 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc3870-db48-47de-a582-998f586580fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:44.393354 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc3870-db48-47de-a582-998f586580fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:44.394425 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdc3870-db48-47de-a582-998f586580fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:44.397271 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdc3870-db48-47de-a582-998f586580fd admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:44.403789 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0237c2c-bd0a-4004-9dcf-387820fe8ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:44.407547 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0237c2c-bd0a-4004-9dcf-387820fe8ced admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:44.408058 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0237c2c-bd0a-4004-9dcf-387820fe8ced admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:44.409343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0237c2c-bd0a-4004-9dcf-387820fe8ced admin admin] PATH_INFO: `/v3/projects/e0f85e9a88ee42779f407c06304405bb/users/4511ccae6d494b9e861ba35eff80c093/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:44.447626 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 996/1961] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:44 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:19:44.452775 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc3870-db48-47de-a582-998f586580fd admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:44.482736 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0237c2c-bd0a-4004-9dcf-387820fe8ced admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0f85e9a88ee42779f407c06304405bb, user_id=4511ccae6d494b9e861ba35eff80c093, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:44.496980 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0237c2c-bd0a-4004-9dcf-387820fe8ced admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:44.540048 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 966/1962] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:44 2022] PUT /identity/v3/projects/e0f85e9a88ee42779f407c06304405bb/users/4511ccae6d494b9e861ba35eff80c093/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:19:44.545435 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0237c2c-bd0a-4004-9dcf-387820fe8ced admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:44.618130 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc3870-db48-47de-a582-998f586580fd admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:44.705332 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0237c2c-bd0a-4004-9dcf-387820fe8ced admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:44.722309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba34ee6-89a4-43f5-ac73-deac91221d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:44.723165 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba34ee6-89a4-43f5-ac73-deac91221d1c admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:44.723553 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba34ee6-89a4-43f5-ac73-deac91221d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:44.723933 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba34ee6-89a4-43f5-ac73-deac91221d1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:44.724755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba34ee6-89a4-43f5-ac73-deac91221d1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:44.726611 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba34ee6-89a4-43f5-ac73-deac91221d1c admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:44.759170 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 967/1963] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:44.760300 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba34ee6-89a4-43f5-ac73-deac91221d1c admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:44.769148 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-cba34ee6-89a4-43f5-ac73-deac91221d1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:44.793441 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb11c3-8994-47e1-b5c3-982f8dda09f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:44.794264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb11c3-8994-47e1-b5c3-982f8dda09f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:44.794553 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb11c3-8994-47e1-b5c3-982f8dda09f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:44.795067 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba34ee6-89a4-43f5-ac73-deac91221d1c admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:44.795581 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb11c3-8994-47e1-b5c3-982f8dda09f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:44.796468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeb11c3-8994-47e1-b5c3-982f8dda09f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:44.799383 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeb11c3-8994-47e1-b5c3-982f8dda09f0 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:44.824661 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa47ae-6589-43c5-b870-789407b52749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:44.825442 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa47ae-6589-43c5-b870-789407b52749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:44.825844 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa47ae-6589-43c5-b870-789407b52749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:44.826160 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa47ae-6589-43c5-b870-789407b52749 admin admin] PATH_INFO: `/v3/projects/7558aa222641481f83c138ad4dca7342` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:44.827925 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa47ae-6589-43c5-b870-789407b52749 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7558aa222641481f83c138ad4dca7342)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:44.830335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa47ae-6589-43c5-b870-789407b52749 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:44.855971 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 997/1964] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:44 2022] POST /identity/v3/users => generated 291 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:44.855971 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb11c3-8994-47e1-b5c3-982f8dda09f0 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:44.877053 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb11c3-8994-47e1-b5c3-982f8dda09f0 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:44.897202 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a4aa98-0fec-4ffe-b968-10b83bae383c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:44.897902 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4aa98-0fec-4ffe-b968-10b83bae383c admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:44.898183 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4aa98-0fec-4ffe-b968-10b83bae383c admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:44.898459 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4aa98-0fec-4ffe-b968-10b83bae383c admin admin] PATH_INFO: `/v3/projects/e0f85e9a88ee42779f407c06304405bb/users/4511ccae6d494b9e861ba35eff80c093/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:44.902646 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a4aa98-0fec-4ffe-b968-10b83bae383c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0f85e9a88ee42779f407c06304405bb, user_id=4511ccae6d494b9e861ba35eff80c093, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:44.908476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a4aa98-0fec-4ffe-b968-10b83bae383c admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:44.945153 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 998/1965] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:44 2022] PUT /identity/v3/projects/e0f85e9a88ee42779f407c06304405bb/users/4511ccae6d494b9e861ba35eff80c093/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:44.951543 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a4aa98-0fec-4ffe-b968-10b83bae383c admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:44.969615 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3baa47ae-6589-43c5-b870-789407b52749 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7558aa222641481f83c138ad4dca7342', 'request_id': 'req-3baa47ae-6589-43c5-b870-789407b52749'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:44.976820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:44.988277 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3baa47ae-6589-43c5-b870-789407b52749 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7558aa222641481f83c138ad4dca7342', 'request_id': 'req-3baa47ae-6589-43c5-b870-789407b52749'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:45.005255 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3baa47ae-6589-43c5-b870-789407b52749 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7558aa222641481f83c138ad4dca7342', 'request_id': 'req-3baa47ae-6589-43c5-b870-789407b52749'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:45.038449 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3baa47ae-6589-43c5-b870-789407b52749 admin admin] The token cache is being invalidate because project 7558aa222641481f83c138ad4dca7342 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:45.038838 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3baa47ae-6589-43c5-b870-789407b52749 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:45.041459 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3baa47ae-6589-43c5-b870-789407b52749 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:45.045199 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 968/1966] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:44 2022] DELETE /identity/v3/projects/7558aa222641481f83c138ad4dca7342 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:45.051264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:45.052267 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62671395-1a12-4f5d-874b-2e085c50af49 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:45.052561 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62671395-1a12-4f5d-874b-2e085c50af49 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:45.053083 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62671395-1a12-4f5d-874b-2e085c50af49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:45.102147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-62671395-1a12-4f5d-874b-2e085c50af49 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:45.108362 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-62671395-1a12-4f5d-874b-2e085c50af49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:45.136392 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a4aa98-0fec-4ffe-b968-10b83bae383c admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:45.150037 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb8d6c-fdd2-4497-b893-4e678f24ae44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:45.151485 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb8d6c-fdd2-4497-b893-4e678f24ae44 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:45.151837 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb8d6c-fdd2-4497-b893-4e678f24ae44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:45.152205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb8d6c-fdd2-4497-b893-4e678f24ae44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:45.153019 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb8d6c-fdd2-4497-b893-4e678f24ae44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:45.154717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb8d6c-fdd2-4497-b893-4e678f24ae44 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:45.200192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 999/1967] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:45.207792 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb8d6c-fdd2-4497-b893-4e678f24ae44 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:45.216281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-27fb8d6c-fdd2-4497-b893-4e678f24ae44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:45.275104 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb8d6c-fdd2-4497-b893-4e678f24ae44 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:45.300522 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ddb46-188f-4876-ad61-cb7fd2b8e283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:45.301377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ddb46-188f-4876-ad61-cb7fd2b8e283 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:45.301490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ddb46-188f-4876-ad61-cb7fd2b8e283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:45.301760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ddb46-188f-4876-ad61-cb7fd2b8e283 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:45.302445 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ddb46-188f-4876-ad61-cb7fd2b8e283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:45.304370 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ddb46-188f-4876-ad61-cb7fd2b8e283 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:45.317699 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 969/1968] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:45.327822 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1f50d-ae08-4417-8ee7-5b40259fbd28 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:45.328060 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1f50d-ae08-4417-8ee7-5b40259fbd28 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:45.328439 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1f50d-ae08-4417-8ee7-5b40259fbd28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:45.347811 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1000/1969] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:45.353210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ddb46-188f-4876-ad61-cb7fd2b8e283 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:45.386346 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-5cc1f50d-ae08-4417-8ee7-5b40259fbd28 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:45.388161 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ddb46-188f-4876-ad61-cb7fd2b8e283 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:45.389124 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-5cc1f50d-ae08-4417-8ee7-5b40259fbd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:45.404745 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5ea19-a69d-4b02-9f84-f447ac8adc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:45.405500 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5ea19-a69d-4b02-9f84-f447ac8adc9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:45.405838 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5ea19-a69d-4b02-9f84-f447ac8adc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:45.406073 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5ea19-a69d-4b02-9f84-f447ac8adc9b admin admin] PATH_INFO: `/v3/projects/3bb6e710ab194e2081453e9bec893275/users/7d30e4ca236d4f99925445705e051b49/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:45.438599 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 970/1970] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:45.440773 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1f50d-ae08-4417-8ee7-5b40259fbd28 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:45.458242 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1f50d-ae08-4417-8ee7-5b40259fbd28 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:45.473476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9bb655-347c-40a1-a4d9-85fe79d58ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:45.473750 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5ea19-a69d-4b02-9f84-f447ac8adc9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bb6e710ab194e2081453e9bec893275, user_id=7d30e4ca236d4f99925445705e051b49, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:45.474377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bb655-347c-40a1-a4d9-85fe79d58ddd admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:45.474763 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bb655-347c-40a1-a4d9-85fe79d58ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:45.475206 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bb655-347c-40a1-a4d9-85fe79d58ddd admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:45.476036 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9bb655-347c-40a1-a4d9-85fe79d58ddd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:19:45.476574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9bb655-347c-40a1-a4d9-85fe79d58ddd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:45.483275 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5ea19-a69d-4b02-9f84-f447ac8adc9b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:45.500304 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9bb655-347c-40a1-a4d9-85fe79d58ddd admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:45.533030 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 971/1971] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:19:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:19:45.534380 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9bb655-347c-40a1-a4d9-85fe79d58ddd admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:45.559599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1001/1972] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:45 2022] PUT /identity/v3/projects/3bb6e710ab194e2081453e9bec893275/users/7d30e4ca236d4f99925445705e051b49/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:45.560045 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9bb655-347c-40a1-a4d9-85fe79d58ddd admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:45.565231 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6c2d6-790b-481a-993d-842e45d5e3dd None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:45.566327 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6c2d6-790b-481a-993d-842e45d5e3dd None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:45.567466 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6c2d6-790b-481a-993d-842e45d5e3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:45.754465 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd16a956-190d-4049-bb09-4d0b75f749f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:45.755467 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16a956-190d-4049-bb09-4d0b75f749f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:45.755873 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16a956-190d-4049-bb09-4d0b75f749f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:45.756254 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16a956-190d-4049-bb09-4d0b75f749f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:45.757039 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd16a956-190d-4049-bb09-4d0b75f749f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:45.761071 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd16a956-190d-4049-bb09-4d0b75f749f6 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:45.773631 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-a0c6c2d6-790b-481a-993d-842e45d5e3dd None None] MFA Rules not processed for user `7d30e4ca236d4f99925445705e051b49`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:45.786487 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a0c6c2d6-790b-481a-993d-842e45d5e3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:45.824530 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 972/1973] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:45 2022] POST /identity/v3/projects => generated 381 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:45.828800 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd16a956-190d-4049-bb09-4d0b75f749f6 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:45.975876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:46.026923 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd16a956-190d-4049-bb09-4d0b75f749f6 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:46.068061 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65f2a3-612e-445d-b67e-ad0540d2f0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:46.068728 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65f2a3-612e-445d-b67e-ad0540d2f0ca admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:46.071310 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65f2a3-612e-445d-b67e-ad0540d2f0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:46.071652 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65f2a3-612e-445d-b67e-ad0540d2f0ca admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:46.072340 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f65f2a3-612e-445d-b67e-ad0540d2f0ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:46.074325 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f65f2a3-612e-445d-b67e-ad0540d2f0ca admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:46.077009 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1002/1974] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 515 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:46.082351 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6c2d6-790b-481a-993d-842e45d5e3dd None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:46.121698 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6c2d6-790b-481a-993d-842e45d5e3dd None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:46.128030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18edc2f-afd5-4ca1-9d2c-ad6255b228cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:46.133017 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18edc2f-afd5-4ca1-9d2c-ad6255b228cf admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:46.133618 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18edc2f-afd5-4ca1-9d2c-ad6255b228cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:46.134155 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18edc2f-afd5-4ca1-9d2c-ad6255b228cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:46.136173 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18edc2f-afd5-4ca1-9d2c-ad6255b228cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:46.139446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18edc2f-afd5-4ca1-9d2c-ad6255b228cf admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:46.170216 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 973/1975] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:45 2022] POST /identity/v3/users => generated 291 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:19:46.171213 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65f2a3-612e-445d-b67e-ad0540d2f0ca admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:46.196540 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65f2a3-612e-445d-b67e-ad0540d2f0ca admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:46.208084 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1003/1976] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:46 2022] POST /identity/v3/projects => generated 381 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:46.209200 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18edc2f-afd5-4ca1-9d2c-ad6255b228cf admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:46.213884 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a624e-9556-414a-a477-45caaa531d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:46.214848 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a624e-9556-414a-a477-45caaa531d68 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:46.215131 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a624e-9556-414a-a477-45caaa531d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:46.216280 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a624e-9556-414a-a477-45caaa531d68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:46.218044 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980a624e-9556-414a-a477-45caaa531d68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:46.222817 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980a624e-9556-414a-a477-45caaa531d68 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:46.260615 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 974/1977] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:46.263044 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a624e-9556-414a-a477-45caaa531d68 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:46.282973 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a624e-9556-414a-a477-45caaa531d68 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:46.383384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18edc2f-afd5-4ca1-9d2c-ad6255b228cf admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:46.393166 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20d495-2a55-4b4a-9102-65e5f8aeca8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:46.393813 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20d495-2a55-4b4a-9102-65e5f8aeca8b admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:46.394098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20d495-2a55-4b4a-9102-65e5f8aeca8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:46.394374 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20d495-2a55-4b4a-9102-65e5f8aeca8b admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:46.395029 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20d495-2a55-4b4a-9102-65e5f8aeca8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:46.397058 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20d495-2a55-4b4a-9102-65e5f8aeca8b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:46.417264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:46.442960 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1004/1978] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:46 2022] POST /identity/v3/users => generated 297 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:46.445487 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20d495-2a55-4b4a-9102-65e5f8aeca8b admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:46.454634 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa7c7a0-32ed-4a45-b1c6-8131aed025e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:46.455418 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa7c7a0-32ed-4a45-b1c6-8131aed025e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:46.455791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa7c7a0-32ed-4a45-b1c6-8131aed025e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:46.456136 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa7c7a0-32ed-4a45-b1c6-8131aed025e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:46.457066 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa7c7a0-32ed-4a45-b1c6-8131aed025e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:46.458854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa7c7a0-32ed-4a45-b1c6-8131aed025e9 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:46.469513 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20d495-2a55-4b4a-9102-65e5f8aeca8b admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:46.483838 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14674a9d-81eb-4065-89e3-f322ef719382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:46.484298 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14674a9d-81eb-4065-89e3-f322ef719382 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:46.489353 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14674a9d-81eb-4065-89e3-f322ef719382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:46.489353 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14674a9d-81eb-4065-89e3-f322ef719382 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:46.489353 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14674a9d-81eb-4065-89e3-f322ef719382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:46.489353 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14674a9d-81eb-4065-89e3-f322ef719382 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:46.489879 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 975/1979] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:46.491400 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa7c7a0-32ed-4a45-b1c6-8131aed025e9 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:46.513743 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1005/1980] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:46.518223 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14674a9d-81eb-4065-89e3-f322ef719382 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:46.520700 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa7c7a0-32ed-4a45-b1c6-8131aed025e9 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:46.531437 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be70fc-dedd-48be-9b6b-5c0eab33460e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:46.533023 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be70fc-dedd-48be-9b6b-5c0eab33460e admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:46.533499 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be70fc-dedd-48be-9b6b-5c0eab33460e admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:46.533993 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be70fc-dedd-48be-9b6b-5c0eab33460e admin admin] PATH_INFO: `/v3/projects/914f907eaa6346559a6ac977f55168a8/users/22dac045558b4f78bb4c561575c2aaa7/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:46.542398 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14674a9d-81eb-4065-89e3-f322ef719382 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:46.555563 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b28424-1689-4838-95c0-7370f444921b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:46.556081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b28424-1689-4838-95c0-7370f444921b admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:46.557753 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b28424-1689-4838-95c0-7370f444921b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:46.558079 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b28424-1689-4838-95c0-7370f444921b admin admin] PATH_INFO: `/v3/projects/4b1b5b431637480a8fc07fe4f541fcc4/users/5dd6c43b501c46c3aca1e671c614f6ec/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:46.576458 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be70fc-dedd-48be-9b6b-5c0eab33460e admin admin] RBAC: Authorizing `identity:create_grant(project_id=914f907eaa6346559a6ac977f55168a8, user_id=22dac045558b4f78bb4c561575c2aaa7, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:46.583764 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be70fc-dedd-48be-9b6b-5c0eab33460e admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:46.597927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b28424-1689-4838-95c0-7370f444921b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b1b5b431637480a8fc07fe4f541fcc4, user_id=5dd6c43b501c46c3aca1e671c614f6ec, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:46.604597 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b28424-1689-4838-95c0-7370f444921b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:46.613227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 976/1981] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:46 2022] PUT /identity/v3/projects/914f907eaa6346559a6ac977f55168a8/users/22dac045558b4f78bb4c561575c2aaa7/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:46.628376 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757355cb-da7c-404a-a199-614066a07b39 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:46.628700 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757355cb-da7c-404a-a199-614066a07b39 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:46.628929 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757355cb-da7c-404a-a199-614066a07b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:46.635775 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1006/1982] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:46 2022] PUT /identity/v3/projects/4b1b5b431637480a8fc07fe4f541fcc4/users/5dd6c43b501c46c3aca1e671c614f6ec/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:46.641296 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b28424-1689-4838-95c0-7370f444921b admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:46.670554 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-757355cb-da7c-404a-a199-614066a07b39 None None] MFA Rules not processed for user `22dac045558b4f78bb4c561575c2aaa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:46.671853 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-757355cb-da7c-404a-a199-614066a07b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:46.769459 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 977/1983] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:46.770834 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757355cb-da7c-404a-a199-614066a07b39 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:46.774855 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b28424-1689-4838-95c0-7370f444921b admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:46.782901 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d91fec-861d-4f02-95dc-0473c0fad5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:46.784105 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d91fec-861d-4f02-95dc-0473c0fad5c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:46.784699 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d91fec-861d-4f02-95dc-0473c0fad5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:46.785141 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d91fec-861d-4f02-95dc-0473c0fad5c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:46.785977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d91fec-861d-4f02-95dc-0473c0fad5c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:46.788302 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d91fec-861d-4f02-95dc-0473c0fad5c0 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:46.798781 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757355cb-da7c-404a-a199-614066a07b39 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:46.812654 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfcb1f1-cd16-4fa4-be81-585dc3ca9e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:46.813849 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfcb1f1-cd16-4fa4-be81-585dc3ca9e51 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:46.814288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfcb1f1-cd16-4fa4-be81-585dc3ca9e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:46.827879 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfcb1f1-cd16-4fa4-be81-585dc3ca9e51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:46.827879 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfcb1f1-cd16-4fa4-be81-585dc3ca9e51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:46.827879 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfcb1f1-cd16-4fa4-be81-585dc3ca9e51 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:46.828137 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1007/1984] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:19:46.832230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d91fec-861d-4f02-95dc-0473c0fad5c0 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:46.845975 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d91fec-861d-4f02-95dc-0473c0fad5c0 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:46.853857 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26bda9-696f-4ca5-83ee-28b526c1048a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:46.854859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26bda9-696f-4ca5-83ee-28b526c1048a admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:46.855303 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26bda9-696f-4ca5-83ee-28b526c1048a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:46.855797 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26bda9-696f-4ca5-83ee-28b526c1048a admin admin] PATH_INFO: `/v3/projects/4b1b5b431637480a8fc07fe4f541fcc4/users/5dd6c43b501c46c3aca1e671c614f6ec/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:46.860838 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26bda9-696f-4ca5-83ee-28b526c1048a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b1b5b431637480a8fc07fe4f541fcc4, user_id=5dd6c43b501c46c3aca1e671c614f6ec, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:46.864450 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26bda9-696f-4ca5-83ee-28b526c1048a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:46.878579 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1008/1985] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:46 2022] PUT /identity/v3/projects/4b1b5b431637480a8fc07fe4f541fcc4/users/5dd6c43b501c46c3aca1e671c614f6ec/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:19:46.889740 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d2002-4e46-4a15-8af7-25dda223e431 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:46.890368 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d2002-4e46-4a15-8af7-25dda223e431 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:46.891176 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d2002-4e46-4a15-8af7-25dda223e431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:46.936833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 978/1986] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:46 2022] POST /identity/v3/projects => generated 381 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:46.941264 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfcb1f1-cd16-4fa4-be81-585dc3ca9e51 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:46.970599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-469d2002-4e46-4a15-8af7-25dda223e431 None None] MFA Rules not processed for user `5dd6c43b501c46c3aca1e671c614f6ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:46.972121 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-469d2002-4e46-4a15-8af7-25dda223e431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:46.975453 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:47.111772 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfcb1f1-cd16-4fa4-be81-585dc3ca9e51 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:47.119570 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b56e89-f04a-47f1-8527-2f952c42c38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:47.120341 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b56e89-f04a-47f1-8527-2f952c42c38c admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:47.120627 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b56e89-f04a-47f1-8527-2f952c42c38c admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:47.121091 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b56e89-f04a-47f1-8527-2f952c42c38c admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:47.122059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b56e89-f04a-47f1-8527-2f952c42c38c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:47.124373 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b56e89-f04a-47f1-8527-2f952c42c38c admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:47.134126 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1009/1987] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:46 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:47.142622 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489aff4-23d6-4f61-875d-7c6b9b28679d None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:47.143155 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489aff4-23d6-4f61-875d-7c6b9b28679d None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:47.145634 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489aff4-23d6-4f61-875d-7c6b9b28679d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:47.170328 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 979/1988] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:46 2022] POST /identity/v3/users => generated 297 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:19:47.174775 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b56e89-f04a-47f1-8527-2f952c42c38c admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:47.196609 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b56e89-f04a-47f1-8527-2f952c42c38c admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:47.204669 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39c23d-cd01-4646-863a-1a80d54d0119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:47.205384 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39c23d-cd01-4646-863a-1a80d54d0119 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:47.205957 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39c23d-cd01-4646-863a-1a80d54d0119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:47.206253 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39c23d-cd01-4646-863a-1a80d54d0119 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:47.206930 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e39c23d-cd01-4646-863a-1a80d54d0119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:47.208839 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e39c23d-cd01-4646-863a-1a80d54d0119 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:47.231705 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-5489aff4-23d6-4f61-875d-7c6b9b28679d None None] MFA Rules not processed for user `4511ccae6d494b9e861ba35eff80c093`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:47.233150 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5489aff4-23d6-4f61-875d-7c6b9b28679d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:47.241945 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 980/1989] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:47.246110 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39c23d-cd01-4646-863a-1a80d54d0119 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:47.250558 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-5e39c23d-cd01-4646-863a-1a80d54d0119 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:47.261750 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:47.388490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1010/1990] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:47.390223 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489aff4-23d6-4f61-875d-7c6b9b28679d None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:47.408909 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489aff4-23d6-4f61-875d-7c6b9b28679d None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:47.415269 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec17a7-a21f-422c-9028-213491ceaa5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:47.416698 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec17a7-a21f-422c-9028-213491ceaa5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:47.416988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec17a7-a21f-422c-9028-213491ceaa5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:47.417360 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec17a7-a21f-422c-9028-213491ceaa5b admin admin] PATH_INFO: `/v3/projects/bf082fd875e64a4e8a909fcccde8ae9b/users/b6e662e82aca487b9ba88b9f3a3b2fb7/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:47.435657 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec17a7-a21f-422c-9028-213491ceaa5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf082fd875e64a4e8a909fcccde8ae9b, user_id=b6e662e82aca487b9ba88b9f3a3b2fb7, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:47.439050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec17a7-a21f-422c-9028-213491ceaa5b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:47.460265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1011/1991] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:47 2022] PUT /identity/v3/projects/bf082fd875e64a4e8a909fcccde8ae9b/users/b6e662e82aca487b9ba88b9f3a3b2fb7/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:47.464238 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec17a7-a21f-422c-9028-213491ceaa5b admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:47.465819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39c23d-cd01-4646-863a-1a80d54d0119 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:47.557599 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ec620-f383-441b-adda-231d704f69f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:47.558215 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ec620-f383-441b-adda-231d704f69f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:47.558502 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ec620-f383-441b-adda-231d704f69f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:47.558869 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ec620-f383-441b-adda-231d704f69f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:47.582925 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec17a7-a21f-422c-9028-213491ceaa5b admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:47.593517 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd95ac08-934b-4749-b9c3-67a7c88178b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:47.595791 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95ac08-934b-4749-b9c3-67a7c88178b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:47.596773 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95ac08-934b-4749-b9c3-67a7c88178b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:47.597369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95ac08-934b-4749-b9c3-67a7c88178b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:47.598243 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd95ac08-934b-4749-b9c3-67a7c88178b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:47.601748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd95ac08-934b-4749-b9c3-67a7c88178b8 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:47.622958 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1012/1992] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:47.624814 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd95ac08-934b-4749-b9c3-67a7c88178b8 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:47.628997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-bd95ac08-934b-4749-b9c3-67a7c88178b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:47.643702 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:47.649581 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd95ac08-934b-4749-b9c3-67a7c88178b8 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:47.654760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79797ca7-8b1b-411b-a3df-91502331168a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:47.655515 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79797ca7-8b1b-411b-a3df-91502331168a service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:47.656684 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79797ca7-8b1b-411b-a3df-91502331168a service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:47.656977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79797ca7-8b1b-411b-a3df-91502331168a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:47.660718 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-79797ca7-8b1b-411b-a3df-91502331168a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:47.662503 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ec620-f383-441b-adda-231d704f69f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:47.664223 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ec620-f383-441b-adda-231d704f69f6 service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:47.681314 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 981/1993] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:47 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:47.695671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ec620-f383-441b-adda-231d704f69f6 service cinder] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:47.724524 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ec620-f383-441b-adda-231d704f69f6 service cinder] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:47.732556 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b058245d-cee4-4439-a185-201fc8026671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:47.733315 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058245d-cee4-4439-a185-201fc8026671 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:47.733853 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058245d-cee4-4439-a185-201fc8026671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:47.734142 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058245d-cee4-4439-a185-201fc8026671 admin admin] PATH_INFO: `/v3/projects/bf082fd875e64a4e8a909fcccde8ae9b/users/b6e662e82aca487b9ba88b9f3a3b2fb7/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:47.739721 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b058245d-cee4-4439-a185-201fc8026671 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf082fd875e64a4e8a909fcccde8ae9b, user_id=b6e662e82aca487b9ba88b9f3a3b2fb7, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:47.751981 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b058245d-cee4-4439-a185-201fc8026671 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:47.764848 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 982/1994] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:47 2022] PUT /identity/v3/projects/bf082fd875e64a4e8a909fcccde8ae9b/users/b6e662e82aca487b9ba88b9f3a3b2fb7/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:47.774537 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79797ca7-8b1b-411b-a3df-91502331168a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:47.774537 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79797ca7-8b1b-411b-a3df-91502331168a service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:47.779744 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af25e5b-19da-480a-927c-279040003b62 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:47.780037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af25e5b-19da-480a-927c-279040003b62 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:47.780319 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af25e5b-19da-480a-927c-279040003b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:47.782907 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1013/1995] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:47.871899 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-0af25e5b-19da-480a-927c-279040003b62 None None] MFA Rules not processed for user `b6e662e82aca487b9ba88b9f3a3b2fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:47.873024 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-0af25e5b-19da-480a-927c-279040003b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:48.061021 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 983/1996] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:47 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:48.212113 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79797ca7-8b1b-411b-a3df-91502331168a service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:48.229174 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79797ca7-8b1b-411b-a3df-91502331168a service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:48.268386 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:48.349059 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53fafd-4e00-4890-951f-377681bce690 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:48.350011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53fafd-4e00-4890-951f-377681bce690 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:48.350448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53fafd-4e00-4890-951f-377681bce690 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:48.350872 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53fafd-4e00-4890-951f-377681bce690 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:48.378690 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca53fafd-4e00-4890-951f-377681bce690 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:48.381247 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca53fafd-4e00-4890-951f-377681bce690 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:48.397446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1014/1997] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:48 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:48.557056 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af25e5b-19da-480a-927c-279040003b62 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:48.568007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af25e5b-19da-480a-927c-279040003b62 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:48.572761 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7e383-274a-47ce-89f0-af4cb8d5d7dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:48.573402 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7e383-274a-47ce-89f0-af4cb8d5d7dd service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:48.573689 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7e383-274a-47ce-89f0-af4cb8d5d7dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:48.573960 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7e383-274a-47ce-89f0-af4cb8d5d7dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:48.656653 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7e383-274a-47ce-89f0-af4cb8d5d7dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:48.659617 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7e383-274a-47ce-89f0-af4cb8d5d7dd service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:48.673273 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 984/1998] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:48 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:48.859351 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53fafd-4e00-4890-951f-377681bce690 service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:48.867376 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ca53fafd-4e00-4890-951f-377681bce690 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:49.029194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53fafd-4e00-4890-951f-377681bce690 service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:49.039037 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb870a-07f1-4b69-a9f5-6a80ee6ae10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:49.039799 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb870a-07f1-4b69-a9f5-6a80ee6ae10b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:49.040716 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb870a-07f1-4b69-a9f5-6a80ee6ae10b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:49.041029 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb870a-07f1-4b69-a9f5-6a80ee6ae10b admin admin] PATH_INFO: `/v3/users/dad0c4050e4348d4aadbd2dbc5d655b9` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:49.042877 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb870a-07f1-4b69-a9f5-6a80ee6ae10b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dad0c4050e4348d4aadbd2dbc5d655b9)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:49.045132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb870a-07f1-4b69-a9f5-6a80ee6ae10b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:49.135336 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7e383-274a-47ce-89f0-af4cb8d5d7dd service cinder] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:49.148054 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7e383-274a-47ce-89f0-af4cb8d5d7dd service cinder] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:49.158308 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacffae5-491d-43e7-94f3-4aee8209dafb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:49.161873 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacffae5-491d-43e7-94f3-4aee8209dafb service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:49.162638 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacffae5-491d-43e7-94f3-4aee8209dafb service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:49.163347 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacffae5-491d-43e7-94f3-4aee8209dafb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:49.166517 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e1fb870a-07f1-4b69-a9f5-6a80ee6ae10b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dad0c4050e4348d4aadbd2dbc5d655b9'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:49.169717 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-cacffae5-491d-43e7-94f3-4aee8209dafb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:49.185626 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e1fb870a-07f1-4b69-a9f5-6a80ee6ae10b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dad0c4050e4348d4aadbd2dbc5d655b9'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:49.186953 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e1fb870a-07f1-4b69-a9f5-6a80ee6ae10b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dad0c4050e4348d4aadbd2dbc5d655b9'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:49.192225 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e1fb870a-07f1-4b69-a9f5-6a80ee6ae10b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dad0c4050e4348d4aadbd2dbc5d655b9'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:49.240561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e1fb870a-07f1-4b69-a9f5-6a80ee6ae10b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dad0c4050e4348d4aadbd2dbc5d655b9'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:49.276713 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1015/1999] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:48 2022] DELETE /identity/v3/users/dad0c4050e4348d4aadbd2dbc5d655b9 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:49.277666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb870a-07f1-4b69-a9f5-6a80ee6ae10b admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:49.279896 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e1fb870a-07f1-4b69-a9f5-6a80ee6ae10b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:49.347940 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:49.353500 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacffae5-491d-43e7-94f3-4aee8209dafb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:49.354802 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:49.359538 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacffae5-491d-43e7-94f3-4aee8209dafb service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:49.387115 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 985/2000] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:49 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:49.394844 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74c887-ef70-40eb-91f1-9a9494e8a528 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:49.395197 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74c887-ef70-40eb-91f1-9a9494e8a528 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:49.395483 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74c887-ef70-40eb-91f1-9a9494e8a528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:49.420762 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb870a-07f1-4b69-a9f5-6a80ee6ae10b admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:49.437042 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05334c80-6dd9-4188-8884-f74e1a6569da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:49.439839 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05334c80-6dd9-4188-8884-f74e1a6569da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:49.440314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05334c80-6dd9-4188-8884-f74e1a6569da admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:49.441382 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05334c80-6dd9-4188-8884-f74e1a6569da admin admin] PATH_INFO: `/v3/users/7d30e4ca236d4f99925445705e051b49` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:49.445852 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05334c80-6dd9-4188-8884-f74e1a6569da admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d30e4ca236d4f99925445705e051b49)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:49.447781 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-af74c887-ef70-40eb-91f1-9a9494e8a528 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:49.454016 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05334c80-6dd9-4188-8884-f74e1a6569da admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:49.459191 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-af74c887-ef70-40eb-91f1-9a9494e8a528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:49.491098 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 986/2001] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:49.492057 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74c887-ef70-40eb-91f1-9a9494e8a528 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:49.497262 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-af74c887-ef70-40eb-91f1-9a9494e8a528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:49.531145 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74c887-ef70-40eb-91f1-9a9494e8a528 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:49.550335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3b516-9ab1-40de-b094-1cab9d7f4003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:49.552491 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3b516-9ab1-40de-b094-1cab9d7f4003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:49.554340 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3b516-9ab1-40de-b094-1cab9d7f4003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:49.555300 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3b516-9ab1-40de-b094-1cab9d7f4003 admin admin] PATH_INFO: `/v3/users/22dac045558b4f78bb4c561575c2aaa7` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:49.557766 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3b516-9ab1-40de-b094-1cab9d7f4003 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22dac045558b4f78bb4c561575c2aaa7)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:49.560369 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3b516-9ab1-40de-b094-1cab9d7f4003 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:49.622324 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-05334c80-6dd9-4188-8884-f74e1a6569da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d30e4ca236d4f99925445705e051b49'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:49.645740 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-05334c80-6dd9-4188-8884-f74e1a6569da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d30e4ca236d4f99925445705e051b49'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:49.646650 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-05334c80-6dd9-4188-8884-f74e1a6569da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d30e4ca236d4f99925445705e051b49'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:49.647302 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-05334c80-6dd9-4188-8884-f74e1a6569da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d30e4ca236d4f99925445705e051b49'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:49.687021 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-05334c80-6dd9-4188-8884-f74e1a6569da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d30e4ca236d4f99925445705e051b49'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:49.699288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-1fe3b516-9ab1-40de-b094-1cab9d7f4003 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22dac045558b4f78bb4c561575c2aaa7'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:49.709875 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1016/2002] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:49 2022] DELETE /identity/v3/users/7d30e4ca236d4f99925445705e051b49 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:49.709875 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05334c80-6dd9-4188-8884-f74e1a6569da admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:49.710126 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-05334c80-6dd9-4188-8884-f74e1a6569da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:49.725636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-1fe3b516-9ab1-40de-b094-1cab9d7f4003 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22dac045558b4f78bb4c561575c2aaa7'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:49.726671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-1fe3b516-9ab1-40de-b094-1cab9d7f4003 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22dac045558b4f78bb4c561575c2aaa7'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:49.758016 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-1fe3b516-9ab1-40de-b094-1cab9d7f4003 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22dac045558b4f78bb4c561575c2aaa7'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:49.771765 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-1fe3b516-9ab1-40de-b094-1cab9d7f4003 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22dac045558b4f78bb4c561575c2aaa7'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:49.789326 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 987/2003] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:49 2022] DELETE /identity/v3/users/22dac045558b4f78bb4c561575c2aaa7 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:49.790352 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3b516-9ab1-40de-b094-1cab9d7f4003 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:49.795198 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-1fe3b516-9ab1-40de-b094-1cab9d7f4003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:49.798656 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05334c80-6dd9-4188-8884-f74e1a6569da admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:49.933702 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c8a794-8f13-4148-b662-394a515acde2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:49.934343 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8a794-8f13-4148-b662-394a515acde2 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:49.934626 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8a794-8f13-4148-b662-394a515acde2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:49.934923 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8a794-8f13-4148-b662-394a515acde2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:49.938283 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-55c8a794-8f13-4148-b662-394a515acde2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:49.960292 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3b516-9ab1-40de-b094-1cab9d7f4003 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:49.968786 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8430269-3670-440a-88e4-18f955ddf1ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:49.969488 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8430269-3670-440a-88e4-18f955ddf1ff service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:49.970268 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8430269-3670-440a-88e4-18f955ddf1ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:49.970546 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8430269-3670-440a-88e4-18f955ddf1ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:49.976622 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-b8430269-3670-440a-88e4-18f955ddf1ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:50.005423 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:50.060754 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c8a794-8f13-4148-b662-394a515acde2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:50.062241 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c8a794-8f13-4148-b662-394a515acde2 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:50.077017 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1017/2004] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:50.078224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f66f5-c7cb-46a6-8738-1d32bc99036e None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:50.078519 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f66f5-c7cb-46a6-8738-1d32bc99036e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:50.079374 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f66f5-c7cb-46a6-8738-1d32bc99036e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:50.103274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-0f0f66f5-c7cb-46a6-8738-1d32bc99036e None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:50.105249 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-0f0f66f5-c7cb-46a6-8738-1d32bc99036e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:50.108693 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8430269-3670-440a-88e4-18f955ddf1ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:50.111499 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8430269-3670-440a-88e4-18f955ddf1ff service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:50.121088 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1018/2005] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:50.121088 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7792414-5bfa-4077-9a46-62533c33913c None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:50.121088 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7792414-5bfa-4077-9a46-62533c33913c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:50.121088 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7792414-5bfa-4077-9a46-62533c33913c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:50.136220 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 988/2006] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:49 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:50.149377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-d7792414-5bfa-4077-9a46-62533c33913c None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:50.150994 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d7792414-5bfa-4077-9a46-62533c33913c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:50.169600 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1019/2007] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:50.177221 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8430269-3670-440a-88e4-18f955ddf1ff service cinder] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:50.178999 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-b8430269-3670-440a-88e4-18f955ddf1ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:50.192282 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8430269-3670-440a-88e4-18f955ddf1ff service cinder] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:50.196610 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e2c9f-43d6-4a8a-bd62-9e65fa1fb44a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:50.197410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e2c9f-43d6-4a8a-bd62-9e65fa1fb44a service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:50.197766 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e2c9f-43d6-4a8a-bd62-9e65fa1fb44a service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:50.198046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e2c9f-43d6-4a8a-bd62-9e65fa1fb44a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:50.211188 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5e2c9f-43d6-4a8a-bd62-9e65fa1fb44a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:50.213158 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5e2c9f-43d6-4a8a-bd62-9e65fa1fb44a service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:50.214843 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7792414-5bfa-4077-9a46-62533c33913c None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:50.218593 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:50.231675 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 989/2008] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:50.232594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7792414-5bfa-4077-9a46-62533c33913c None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:50.242559 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8331c8c-2d3d-476f-9208-df087d92ce97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:50.243207 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8331c8c-2d3d-476f-9208-df087d92ce97 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:50.243663 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8331c8c-2d3d-476f-9208-df087d92ce97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:50.244009 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8331c8c-2d3d-476f-9208-df087d92ce97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:50.257765 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8331c8c-2d3d-476f-9208-df087d92ce97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:50.259503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8331c8c-2d3d-476f-9208-df087d92ce97 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:50.273385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1020/2009] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:19:50.685191 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e2c9f-43d6-4a8a-bd62-9e65fa1fb44a service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:50.686821 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-ac5e2c9f-43d6-4a8a-bd62-9e65fa1fb44a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:50.701824 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e2c9f-43d6-4a8a-bd62-9e65fa1fb44a service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:50.709718 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8c938-b51e-40ef-866b-d127ce01eeb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:50.710670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8c938-b51e-40ef-866b-d127ce01eeb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:50.711119 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8c938-b51e-40ef-866b-d127ce01eeb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:50.711572 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8c938-b51e-40ef-866b-d127ce01eeb9 admin admin] PATH_INFO: `/v3/projects/88fa24294afe4dce87c5263b482c5679` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:50.713818 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c8c938-b51e-40ef-866b-d127ce01eeb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88fa24294afe4dce87c5263b482c5679)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:50.716072 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c8c938-b51e-40ef-866b-d127ce01eeb9 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:50.794734 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-69c8c938-b51e-40ef-866b-d127ce01eeb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88fa24294afe4dce87c5263b482c5679', 'request_id': 'req-69c8c938-b51e-40ef-866b-d127ce01eeb9'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:50.806519 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-69c8c938-b51e-40ef-866b-d127ce01eeb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88fa24294afe4dce87c5263b482c5679', 'request_id': 'req-69c8c938-b51e-40ef-866b-d127ce01eeb9'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:50.816024 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-69c8c938-b51e-40ef-866b-d127ce01eeb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88fa24294afe4dce87c5263b482c5679', 'request_id': 'req-69c8c938-b51e-40ef-866b-d127ce01eeb9'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:50.838370 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-69c8c938-b51e-40ef-866b-d127ce01eeb9 admin admin] The token cache is being invalidate because project 88fa24294afe4dce87c5263b482c5679 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:50.838881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-69c8c938-b51e-40ef-866b-d127ce01eeb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:50.840161 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-69c8c938-b51e-40ef-866b-d127ce01eeb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:50.842885 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 990/2010] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:50 2022] DELETE /identity/v3/projects/88fa24294afe4dce87c5263b482c5679 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:50.848676 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8331c8c-2d3d-476f-9208-df087d92ce97 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:50.869837 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8331c8c-2d3d-476f-9208-df087d92ce97 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:51.029677 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f288405-a6c1-4344-87d7-5357f020a629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:51.030366 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f288405-a6c1-4344-87d7-5357f020a629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:51.030598 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f288405-a6c1-4344-87d7-5357f020a629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:51.030878 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f288405-a6c1-4344-87d7-5357f020a629 admin admin] PATH_INFO: `/v3/users/5cf0980412b54ba1960017841ab87b8f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:51.033270 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f288405-a6c1-4344-87d7-5357f020a629 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cf0980412b54ba1960017841ab87b8f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:51.037710 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f288405-a6c1-4344-87d7-5357f020a629 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:51.084189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8c938-b51e-40ef-866b-d127ce01eeb9 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:51.085675 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-69c8c938-b51e-40ef-866b-d127ce01eeb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:51.102372 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8c938-b51e-40ef-866b-d127ce01eeb9 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:51.117477 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd553f1-4f47-462c-b51c-dc4758c2ca18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:51.120089 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd553f1-4f47-462c-b51c-dc4758c2ca18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:51.121641 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd553f1-4f47-462c-b51c-dc4758c2ca18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:51.121769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd553f1-4f47-462c-b51c-dc4758c2ca18 admin admin] PATH_INFO: `/v3/projects/3bb6e710ab194e2081453e9bec893275` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:51.124461 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd553f1-4f47-462c-b51c-dc4758c2ca18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bb6e710ab194e2081453e9bec893275)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:51.130639 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd553f1-4f47-462c-b51c-dc4758c2ca18 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:51.140132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0f288405-a6c1-4344-87d7-5357f020a629 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cf0980412b54ba1960017841ab87b8f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:51.152986 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0f288405-a6c1-4344-87d7-5357f020a629 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cf0980412b54ba1960017841ab87b8f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:51.154487 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0f288405-a6c1-4344-87d7-5357f020a629 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cf0980412b54ba1960017841ab87b8f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:51.158119 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0f288405-a6c1-4344-87d7-5357f020a629 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cf0980412b54ba1960017841ab87b8f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:51.178306 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0f288405-a6c1-4344-87d7-5357f020a629 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cf0980412b54ba1960017841ab87b8f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:51.209533 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1021/2011] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:50 2022] DELETE /identity/v3/users/5cf0980412b54ba1960017841ab87b8f => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:51.228798 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-fcd553f1-4f47-462c-b51c-dc4758c2ca18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bb6e710ab194e2081453e9bec893275', 'request_id': 'req-fcd553f1-4f47-462c-b51c-dc4758c2ca18'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:51.240127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-fcd553f1-4f47-462c-b51c-dc4758c2ca18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bb6e710ab194e2081453e9bec893275', 'request_id': 'req-fcd553f1-4f47-462c-b51c-dc4758c2ca18'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:51.254034 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-fcd553f1-4f47-462c-b51c-dc4758c2ca18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bb6e710ab194e2081453e9bec893275', 'request_id': 'req-fcd553f1-4f47-462c-b51c-dc4758c2ca18'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:51.269518 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-fcd553f1-4f47-462c-b51c-dc4758c2ca18 admin admin] The token cache is being invalidate because project 3bb6e710ab194e2081453e9bec893275 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:51.269777 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-fcd553f1-4f47-462c-b51c-dc4758c2ca18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:51.270589 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-fcd553f1-4f47-462c-b51c-dc4758c2ca18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:51.276038 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 991/2012] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:51 2022] DELETE /identity/v3/projects/3bb6e710ab194e2081453e9bec893275 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:19:51.277931 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f288405-a6c1-4344-87d7-5357f020a629 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:51.281326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-0f288405-a6c1-4344-87d7-5357f020a629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:51.329833 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:51.393998 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd553f1-4f47-462c-b51c-dc4758c2ca18 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:51.396123 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-fcd553f1-4f47-462c-b51c-dc4758c2ca18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:51.488225 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f288405-a6c1-4344-87d7-5357f020a629 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:51.503893 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e488ff4-b85e-4c47-90d2-f5e85cccd751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:51.505919 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e488ff4-b85e-4c47-90d2-f5e85cccd751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:51.506402 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e488ff4-b85e-4c47-90d2-f5e85cccd751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:51.506826 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e488ff4-b85e-4c47-90d2-f5e85cccd751 admin admin] PATH_INFO: `/v3/users/5dd6c43b501c46c3aca1e671c614f6ec` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:51.508475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e488ff4-b85e-4c47-90d2-f5e85cccd751 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dd6c43b501c46c3aca1e671c614f6ec)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:51.521595 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e488ff4-b85e-4c47-90d2-f5e85cccd751 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:51.678499 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd553f1-4f47-462c-b51c-dc4758c2ca18 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:51.697535 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260fe015-a0ff-490f-8ad4-b53656f6ab34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:51.698519 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fe015-a0ff-490f-8ad4-b53656f6ab34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:51.698960 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fe015-a0ff-490f-8ad4-b53656f6ab34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:51.699376 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fe015-a0ff-490f-8ad4-b53656f6ab34 admin admin] PATH_INFO: `/v3/projects/914f907eaa6346559a6ac977f55168a8` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:51.701289 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260fe015-a0ff-490f-8ad4-b53656f6ab34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=914f907eaa6346559a6ac977f55168a8)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:51.705248 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260fe015-a0ff-490f-8ad4-b53656f6ab34 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:51.718271 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:51.728189 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2e488ff4-b85e-4c47-90d2-f5e85cccd751 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dd6c43b501c46c3aca1e671c614f6ec'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:51.756969 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2e488ff4-b85e-4c47-90d2-f5e85cccd751 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dd6c43b501c46c3aca1e671c614f6ec'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:51.758036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2e488ff4-b85e-4c47-90d2-f5e85cccd751 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dd6c43b501c46c3aca1e671c614f6ec'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:51.758791 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2e488ff4-b85e-4c47-90d2-f5e85cccd751 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dd6c43b501c46c3aca1e671c614f6ec'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:51.792065 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2e488ff4-b85e-4c47-90d2-f5e85cccd751 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dd6c43b501c46c3aca1e671c614f6ec'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:51.815464 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-260fe015-a0ff-490f-8ad4-b53656f6ab34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '914f907eaa6346559a6ac977f55168a8', 'request_id': 'req-260fe015-a0ff-490f-8ad4-b53656f6ab34'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:51.835398 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-260fe015-a0ff-490f-8ad4-b53656f6ab34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '914f907eaa6346559a6ac977f55168a8', 'request_id': 'req-260fe015-a0ff-490f-8ad4-b53656f6ab34'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:51.863792 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-260fe015-a0ff-490f-8ad4-b53656f6ab34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '914f907eaa6346559a6ac977f55168a8', 'request_id': 'req-260fe015-a0ff-490f-8ad4-b53656f6ab34'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:51.866278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1022/2013] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:51 2022] DELETE /identity/v3/users/5dd6c43b501c46c3aca1e671c614f6ec => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:51.890489 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-260fe015-a0ff-490f-8ad4-b53656f6ab34 admin admin] The token cache is being invalidate because project 914f907eaa6346559a6ac977f55168a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:51.890994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-260fe015-a0ff-490f-8ad4-b53656f6ab34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:51.892103 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-260fe015-a0ff-490f-8ad4-b53656f6ab34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:51.894300 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 992/2014] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:51 2022] DELETE /identity/v3/projects/914f907eaa6346559a6ac977f55168a8 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:51.898147 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e488ff4-b85e-4c47-90d2-f5e85cccd751 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:51.900257 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2e488ff4-b85e-4c47-90d2-f5e85cccd751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:51.964459 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260fe015-a0ff-490f-8ad4-b53656f6ab34 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:51.966538 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-260fe015-a0ff-490f-8ad4-b53656f6ab34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:52.074810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e488ff4-b85e-4c47-90d2-f5e85cccd751 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:52.094627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113bf371-27a6-49c1-9e1f-852f151213bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:52.095310 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113bf371-27a6-49c1-9e1f-852f151213bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:52.096030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113bf371-27a6-49c1-9e1f-852f151213bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:52.096323 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113bf371-27a6-49c1-9e1f-852f151213bb admin admin] PATH_INFO: `/v3/users/b6e662e82aca487b9ba88b9f3a3b2fb7` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:52.100269 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113bf371-27a6-49c1-9e1f-852f151213bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6e662e82aca487b9ba88b9f3a3b2fb7)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:52.104165 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113bf371-27a6-49c1-9e1f-852f151213bb admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:52.124579 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260fe015-a0ff-490f-8ad4-b53656f6ab34 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:52.133006 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f88fcb-c1a8-4322-9d42-0365d46ca85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:52.133779 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f88fcb-c1a8-4322-9d42-0365d46ca85d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:52.134238 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f88fcb-c1a8-4322-9d42-0365d46ca85d admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:52.134593 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f88fcb-c1a8-4322-9d42-0365d46ca85d admin admin] PATH_INFO: `/v3/projects/e776245c799148588827e227c754d0d7` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:52.137307 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f88fcb-c1a8-4322-9d42-0365d46ca85d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e776245c799148588827e227c754d0d7)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:52.142301 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f88fcb-c1a8-4322-9d42-0365d46ca85d admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:52.229757 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:52.245619 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-113bf371-27a6-49c1-9e1f-852f151213bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6e662e82aca487b9ba88b9f3a3b2fb7'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:52.267702 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-113bf371-27a6-49c1-9e1f-852f151213bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6e662e82aca487b9ba88b9f3a3b2fb7'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:52.270864 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-113bf371-27a6-49c1-9e1f-852f151213bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6e662e82aca487b9ba88b9f3a3b2fb7'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:52.272331 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-113bf371-27a6-49c1-9e1f-852f151213bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6e662e82aca487b9ba88b9f3a3b2fb7'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:52.276661 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-57f88fcb-c1a8-4322-9d42-0365d46ca85d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e776245c799148588827e227c754d0d7', 'request_id': 'req-57f88fcb-c1a8-4322-9d42-0365d46ca85d'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:52.294790 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-57f88fcb-c1a8-4322-9d42-0365d46ca85d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e776245c799148588827e227c754d0d7', 'request_id': 'req-57f88fcb-c1a8-4322-9d42-0365d46ca85d'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:52.310582 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-113bf371-27a6-49c1-9e1f-852f151213bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6e662e82aca487b9ba88b9f3a3b2fb7'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:52.312800 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-57f88fcb-c1a8-4322-9d42-0365d46ca85d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e776245c799148588827e227c754d0d7', 'request_id': 'req-57f88fcb-c1a8-4322-9d42-0365d46ca85d'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:52.329972 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-57f88fcb-c1a8-4322-9d42-0365d46ca85d admin admin] The token cache is being invalidate because project e776245c799148588827e227c754d0d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:52.330329 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-57f88fcb-c1a8-4322-9d42-0365d46ca85d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:52.331369 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-57f88fcb-c1a8-4322-9d42-0365d46ca85d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:52.338536 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 993/2015] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:51 2022] DELETE /identity/v3/projects/e776245c799148588827e227c754d0d7 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:19:52.339649 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1023/2016] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:51 2022] DELETE /identity/v3/users/b6e662e82aca487b9ba88b9f3a3b2fb7 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:52.343136 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6eb79a-54a3-4b9f-8c0e-c0e2cb163cc2 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:52.343458 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6eb79a-54a3-4b9f-8c0e-c0e2cb163cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:52.343872 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6eb79a-54a3-4b9f-8c0e-c0e2cb163cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:52.385403 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-be6eb79a-54a3-4b9f-8c0e-c0e2cb163cc2 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:52.393064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-be6eb79a-54a3-4b9f-8c0e-c0e2cb163cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:52.432713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:52.497215 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 994/2017] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:52.505387 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ad72c-0dff-4a5e-b74a-8733a16df76e None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:52.505942 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ad72c-0dff-4a5e-b74a-8733a16df76e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:52.506398 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ad72c-0dff-4a5e-b74a-8733a16df76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:52.532955 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-221ad72c-0dff-4a5e-b74a-8733a16df76e None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:52.534543 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-221ad72c-0dff-4a5e-b74a-8733a16df76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:52.545703 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1024/2018] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:52.551591 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6eb79a-54a3-4b9f-8c0e-c0e2cb163cc2 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:52.564782 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6eb79a-54a3-4b9f-8c0e-c0e2cb163cc2 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:52.582031 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b605b1c0-8990-49fb-a917-bb2a48dda603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:52.583102 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605b1c0-8990-49fb-a917-bb2a48dda603 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:52.583276 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605b1c0-8990-49fb-a917-bb2a48dda603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:52.583667 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605b1c0-8990-49fb-a917-bb2a48dda603 admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:52.584450 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b605b1c0-8990-49fb-a917-bb2a48dda603 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:19:52.584855 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b605b1c0-8990-49fb-a917-bb2a48dda603 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:52.586975 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b605b1c0-8990-49fb-a917-bb2a48dda603 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:52.603679 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 995/2019] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:19:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:19:52.608061 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ad72c-0dff-4a5e-b74a-8733a16df76e None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:52.621540 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ad72c-0dff-4a5e-b74a-8733a16df76e None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:52.640816 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a344c0-aa86-482e-ad96-77b3bcfe21e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:52.643880 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a344c0-aa86-482e-ad96-77b3bcfe21e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:52.646175 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a344c0-aa86-482e-ad96-77b3bcfe21e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:52.648829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a344c0-aa86-482e-ad96-77b3bcfe21e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:52.652579 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a344c0-aa86-482e-ad96-77b3bcfe21e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:52.660785 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a344c0-aa86-482e-ad96-77b3bcfe21e6 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:52.723905 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b605b1c0-8990-49fb-a917-bb2a48dda603 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:52.726895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-b605b1c0-8990-49fb-a917-bb2a48dda603 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:52.743328 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1025/2020] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:52 2022] POST /identity/v3/projects => generated 375 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:52.748070 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a344c0-aa86-482e-ad96-77b3bcfe21e6 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:52.775783 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b605b1c0-8990-49fb-a917-bb2a48dda603 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:52.913418 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f4b26-e8c0-4d30-8f6a-d6356ba32fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:52.914084 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f4b26-e8c0-4d30-8f6a-d6356ba32fb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:52.914363 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f4b26-e8c0-4d30-8f6a-d6356ba32fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:52.914636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f4b26-e8c0-4d30-8f6a-d6356ba32fb9 admin admin] PATH_INFO: `/v3/projects/4b1b5b431637480a8fc07fe4f541fcc4` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:52.916571 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f4b26-e8c0-4d30-8f6a-d6356ba32fb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b1b5b431637480a8fc07fe4f541fcc4)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:52.918543 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f4b26-e8c0-4d30-8f6a-d6356ba32fb9 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:52.966181 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a344c0-aa86-482e-ad96-77b3bcfe21e6 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:52.973018 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d13356-2bf6-4102-aa89-6cba695a51f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:52.974291 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d13356-2bf6-4102-aa89-6cba695a51f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:52.974580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d13356-2bf6-4102-aa89-6cba695a51f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:52.974852 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d13356-2bf6-4102-aa89-6cba695a51f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:52.975507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d13356-2bf6-4102-aa89-6cba695a51f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:52.977897 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d13356-2bf6-4102-aa89-6cba695a51f1 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:53.007157 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-1c5f4b26-e8c0-4d30-8f6a-d6356ba32fb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b1b5b431637480a8fc07fe4f541fcc4', 'request_id': 'req-1c5f4b26-e8c0-4d30-8f6a-d6356ba32fb9'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:53.015570 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1026/2021] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:52 2022] POST /identity/v3/users => generated 288 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:53.018147 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d13356-2bf6-4102-aa89-6cba695a51f1 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:53.021093 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-1c5f4b26-e8c0-4d30-8f6a-d6356ba32fb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b1b5b431637480a8fc07fe4f541fcc4', 'request_id': 'req-1c5f4b26-e8c0-4d30-8f6a-d6356ba32fb9'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:53.030387 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-1c5f4b26-e8c0-4d30-8f6a-d6356ba32fb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b1b5b431637480a8fc07fe4f541fcc4', 'request_id': 'req-1c5f4b26-e8c0-4d30-8f6a-d6356ba32fb9'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:53.030922 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d13356-2bf6-4102-aa89-6cba695a51f1 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:53.069336 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-1c5f4b26-e8c0-4d30-8f6a-d6356ba32fb9 admin admin] The token cache is being invalidate because project 4b1b5b431637480a8fc07fe4f541fcc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:53.069732 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-1c5f4b26-e8c0-4d30-8f6a-d6356ba32fb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:53.071283 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-1c5f4b26-e8c0-4d30-8f6a-d6356ba32fb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:53.083253 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 996/2022] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:52 2022] DELETE /identity/v3/projects/4b1b5b431637480a8fc07fe4f541fcc4 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:19:53.085774 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30206534-7bce-4d56-82b4-1a6e0ddec778 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:53.086701 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30206534-7bce-4d56-82b4-1a6e0ddec778 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:53.086701 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30206534-7bce-4d56-82b4-1a6e0ddec778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:53.118598 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-30206534-7bce-4d56-82b4-1a6e0ddec778 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:53.120275 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-30206534-7bce-4d56-82b4-1a6e0ddec778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:53.142454 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 997/2023] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:53.142454 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30206534-7bce-4d56-82b4-1a6e0ddec778 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:53.142454 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-30206534-7bce-4d56-82b4-1a6e0ddec778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:53.143959 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caffbaf-054a-4f6a-86c3-4f83717136a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:53.145038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caffbaf-054a-4f6a-86c3-4f83717136a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:53.145395 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caffbaf-054a-4f6a-86c3-4f83717136a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:53.146516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caffbaf-054a-4f6a-86c3-4f83717136a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:53.146516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caffbaf-054a-4f6a-86c3-4f83717136a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:53.150858 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caffbaf-054a-4f6a-86c3-4f83717136a5 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:53.167715 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1027/2024] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:19:53.168910 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30206534-7bce-4d56-82b4-1a6e0ddec778 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:53.172146 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a0a71-64ae-442d-87f7-56f4858a2277 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:53.172555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a0a71-64ae-442d-87f7-56f4858a2277 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:53.172868 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a0a71-64ae-442d-87f7-56f4858a2277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:53.175596 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e70fa3-8d12-4907-84f1-d0b3bc3cb9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:53.176558 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e70fa3-8d12-4907-84f1-d0b3bc3cb9ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:53.176947 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e70fa3-8d12-4907-84f1-d0b3bc3cb9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:53.177256 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e70fa3-8d12-4907-84f1-d0b3bc3cb9ab admin admin] PATH_INFO: `/v3/projects/bf082fd875e64a4e8a909fcccde8ae9b` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:53.178539 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e70fa3-8d12-4907-84f1-d0b3bc3cb9ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf082fd875e64a4e8a909fcccde8ae9b)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:53.180571 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e70fa3-8d12-4907-84f1-d0b3bc3cb9ab admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:53.197108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-2a3a0a71-64ae-442d-87f7-56f4858a2277 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:53.207490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2a3a0a71-64ae-442d-87f7-56f4858a2277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:53.212009 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1028/2025] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:53.212009 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a0a71-64ae-442d-87f7-56f4858a2277 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:53.212009 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2a3a0a71-64ae-442d-87f7-56f4858a2277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:53.225954 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a0a71-64ae-442d-87f7-56f4858a2277 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:53.232763 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ca384-f550-47ea-970c-d7c4afc5954b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:53.233496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ca384-f550-47ea-970c-d7c4afc5954b admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:53.233863 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ca384-f550-47ea-970c-d7c4afc5954b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:53.234309 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ca384-f550-47ea-970c-d7c4afc5954b admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:53.234943 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195ca384-f550-47ea-970c-d7c4afc5954b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:53.236847 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195ca384-f550-47ea-970c-d7c4afc5954b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:53.239383 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:53.254305 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:53.263985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1029/2026] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:53.263985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ca384-f550-47ea-970c-d7c4afc5954b admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:53.273768 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-c1e70fa3-8d12-4907-84f1-d0b3bc3cb9ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf082fd875e64a4e8a909fcccde8ae9b', 'request_id': 'req-c1e70fa3-8d12-4907-84f1-d0b3bc3cb9ab'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:53.287988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-c1e70fa3-8d12-4907-84f1-d0b3bc3cb9ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf082fd875e64a4e8a909fcccde8ae9b', 'request_id': 'req-c1e70fa3-8d12-4907-84f1-d0b3bc3cb9ab'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:53.288949 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ca384-f550-47ea-970c-d7c4afc5954b admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:53.299774 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-c1e70fa3-8d12-4907-84f1-d0b3bc3cb9ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf082fd875e64a4e8a909fcccde8ae9b', 'request_id': 'req-c1e70fa3-8d12-4907-84f1-d0b3bc3cb9ab'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:53.316986 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-c1e70fa3-8d12-4907-84f1-d0b3bc3cb9ab admin admin] The token cache is being invalidate because project bf082fd875e64a4e8a909fcccde8ae9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:53.317290 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-c1e70fa3-8d12-4907-84f1-d0b3bc3cb9ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:53.318022 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-c1e70fa3-8d12-4907-84f1-d0b3bc3cb9ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:53.319627 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 998/2027] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:53 2022] DELETE /identity/v3/projects/bf082fd875e64a4e8a909fcccde8ae9b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:53.320444 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e70fa3-8d12-4907-84f1-d0b3bc3cb9ab admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:53.322046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-c1e70fa3-8d12-4907-84f1-d0b3bc3cb9ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:53.402266 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011722ec-e388-488d-a05f-679c01c3a27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:53.403263 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011722ec-e388-488d-a05f-679c01c3a27f admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:53.403787 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011722ec-e388-488d-a05f-679c01c3a27f admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:53.404235 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011722ec-e388-488d-a05f-679c01c3a27f admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:53.405230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011722ec-e388-488d-a05f-679c01c3a27f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:19:53.405684 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011722ec-e388-488d-a05f-679c01c3a27f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:53.409358 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011722ec-e388-488d-a05f-679c01c3a27f admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:53.426338 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1030/2028] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:19:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:19:53.427826 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af8ca9-0c0a-4a18-91b0-521e71f7bc7f None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:53.428172 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af8ca9-0c0a-4a18-91b0-521e71f7bc7f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:53.428504 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af8ca9-0c0a-4a18-91b0-521e71f7bc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:53.445193 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e70fa3-8d12-4907-84f1-d0b3bc3cb9ab admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:53.456104 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-16af8ca9-0c0a-4a18-91b0-521e71f7bc7f None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:53.458631 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-16af8ca9-0c0a-4a18-91b0-521e71f7bc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:53.473677 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a242890-997d-4f17-aafc-fe114f4d2d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:53.474782 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a242890-997d-4f17-aafc-fe114f4d2d73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:53.475128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a242890-997d-4f17-aafc-fe114f4d2d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:53.475799 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a242890-997d-4f17-aafc-fe114f4d2d73 admin admin] PATH_INFO: `/v3/projects/a0e1ac9f3a274ba2b2469dbdc2ebf2e5/users/7d82ba9bce3546e682bcacdcd8fb1ed5/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:53.494531 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a242890-997d-4f17-aafc-fe114f4d2d73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0e1ac9f3a274ba2b2469dbdc2ebf2e5, user_id=7d82ba9bce3546e682bcacdcd8fb1ed5, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:53.497666 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a242890-997d-4f17-aafc-fe114f4d2d73 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:53.504936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1031/2029] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:53.506136 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af8ca9-0c0a-4a18-91b0-521e71f7bc7f None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:53.508202 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-16af8ca9-0c0a-4a18-91b0-521e71f7bc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:53.520655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 999/2030] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:53 2022] PUT /identity/v3/projects/a0e1ac9f3a274ba2b2469dbdc2ebf2e5/users/7d82ba9bce3546e682bcacdcd8fb1ed5/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:53.527642 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb08bd0-80e1-4a2d-a657-ee68b6910fe8 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:53.527642 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb08bd0-80e1-4a2d-a657-ee68b6910fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:53.527642 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb08bd0-80e1-4a2d-a657-ee68b6910fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:53.528023 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af8ca9-0c0a-4a18-91b0-521e71f7bc7f None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:53.551351 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-ccb08bd0-80e1-4a2d-a657-ee68b6910fe8 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:53.552808 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-ccb08bd0-80e1-4a2d-a657-ee68b6910fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:53.614954 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f419f8-cf9f-41f5-9198-9b9f26ba798a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:53.615762 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f419f8-cf9f-41f5-9198-9b9f26ba798a admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:53.616151 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f419f8-cf9f-41f5-9198-9b9f26ba798a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:53.616433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f419f8-cf9f-41f5-9198-9b9f26ba798a admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:53.617260 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f419f8-cf9f-41f5-9198-9b9f26ba798a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:53.619638 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f419f8-cf9f-41f5-9198-9b9f26ba798a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:53.643960 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1000/2031] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:53.645624 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85544c35-c76d-42b2-babb-b1720510a2d5 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:53.646180 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85544c35-c76d-42b2-babb-b1720510a2d5 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:53.646765 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85544c35-c76d-42b2-babb-b1720510a2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:53.668303 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1032/2032] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:53 2022] POST /identity/v3/projects => generated 383 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:53.670141 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f419f8-cf9f-41f5-9198-9b9f26ba798a admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:53.686919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-85544c35-c76d-42b2-babb-b1720510a2d5 None None] MFA Rules not processed for user `7d82ba9bce3546e682bcacdcd8fb1ed5`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:53.689384 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-85544c35-c76d-42b2-babb-b1720510a2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:53.742912 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:53.772577 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f419f8-cf9f-41f5-9198-9b9f26ba798a admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:53.778710 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4496d-3ac9-4a71-a943-6254c1a3516d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:53.779430 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4496d-3ac9-4a71-a943-6254c1a3516d admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:53.779812 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4496d-3ac9-4a71-a943-6254c1a3516d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:53.780176 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4496d-3ac9-4a71-a943-6254c1a3516d admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:53.780911 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4496d-3ac9-4a71-a943-6254c1a3516d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:19:53.781311 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4496d-3ac9-4a71-a943-6254c1a3516d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:53.783028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4496d-3ac9-4a71-a943-6254c1a3516d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:53.808810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1033/2033] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:19:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:19:53.809705 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1001/2034] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:53.809794 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4496d-3ac9-4a71-a943-6254c1a3516d admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:53.810484 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85544c35-c76d-42b2-babb-b1720510a2d5 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:53.819068 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:53.825121 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4496d-3ac9-4a71-a943-6254c1a3516d admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:53.827379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85544c35-c76d-42b2-babb-b1720510a2d5 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:53.834121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8fca5b-649a-47b1-96cd-cec6cdf083cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:53.834405 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d5072-4f04-4f4c-8aea-3a463aac1239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:53.834750 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fca5b-649a-47b1-96cd-cec6cdf083cf admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:53.835028 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fca5b-649a-47b1-96cd-cec6cdf083cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:53.835299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fca5b-649a-47b1-96cd-cec6cdf083cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:53.835588 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d5072-4f04-4f4c-8aea-3a463aac1239 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:53.836002 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8fca5b-649a-47b1-96cd-cec6cdf083cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:53.836128 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d5072-4f04-4f4c-8aea-3a463aac1239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:53.836436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d5072-4f04-4f4c-8aea-3a463aac1239 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:53.837302 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d5072-4f04-4f4c-8aea-3a463aac1239 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:53.838146 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8fca5b-649a-47b1-96cd-cec6cdf083cf admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:53.840105 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d5072-4f04-4f4c-8aea-3a463aac1239 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:53.891689 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1034/2035] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:53 2022] POST /identity/v3/users => generated 292 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:53.893273 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d5072-4f04-4f4c-8aea-3a463aac1239 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:53.897028 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1002/2036] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:53 2022] POST /identity/v3/projects => generated 367 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:19:53.898917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8fca5b-649a-47b1-96cd-cec6cdf083cf admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:53.911258 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d5072-4f04-4f4c-8aea-3a463aac1239 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:53.995105 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562dc12e-86ac-49b0-8560-a7420776bd8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:53.995944 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562dc12e-86ac-49b0-8560-a7420776bd8e admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:53.996305 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562dc12e-86ac-49b0-8560-a7420776bd8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:53.996664 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562dc12e-86ac-49b0-8560-a7420776bd8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:53.997634 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562dc12e-86ac-49b0-8560-a7420776bd8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:53.999484 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562dc12e-86ac-49b0-8560-a7420776bd8e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:54.016863 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8fca5b-649a-47b1-96cd-cec6cdf083cf admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:54.023988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559bb072-34af-478a-bee6-a577ebc0a604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:54.025107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559bb072-34af-478a-bee6-a577ebc0a604 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:54.025598 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559bb072-34af-478a-bee6-a577ebc0a604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:54.025832 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559bb072-34af-478a-bee6-a577ebc0a604 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:54.026664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559bb072-34af-478a-bee6-a577ebc0a604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:54.028963 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559bb072-34af-478a-bee6-a577ebc0a604 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:54.041355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1035/2037] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:53 2022] POST /identity/v3/projects => generated 375 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:54.041355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562dc12e-86ac-49b0-8560-a7420776bd8e admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:54.048538 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1003/2038] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:54.048538 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559bb072-34af-478a-bee6-a577ebc0a604 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:54.058234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559bb072-34af-478a-bee6-a577ebc0a604 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:54.111411 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562dc12e-86ac-49b0-8560-a7420776bd8e admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:54.119936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758d8e8-07cc-4657-b0d4-6ecad56e0f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:54.120958 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758d8e8-07cc-4657-b0d4-6ecad56e0f2a admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:54.121439 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758d8e8-07cc-4657-b0d4-6ecad56e0f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:54.121995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758d8e8-07cc-4657-b0d4-6ecad56e0f2a admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:54.123108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4758d8e8-07cc-4657-b0d4-6ecad56e0f2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:54.124876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be70e0c-8a41-47a3-844e-711ff3e5fce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:54.125501 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be70e0c-8a41-47a3-844e-711ff3e5fce6 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:54.125818 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be70e0c-8a41-47a3-844e-711ff3e5fce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:54.126097 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be70e0c-8a41-47a3-844e-711ff3e5fce6 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:54.126749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be70e0c-8a41-47a3-844e-711ff3e5fce6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:54.126938 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4758d8e8-07cc-4657-b0d4-6ecad56e0f2a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:54.128774 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be70e0c-8a41-47a3-844e-711ff3e5fce6 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:54.156105 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1004/2039] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:54 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:54.158123 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be70e0c-8a41-47a3-844e-711ff3e5fce6 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:54.161739 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1036/2040] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:54 2022] POST /identity/v3/users => generated 284 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:54.162628 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758d8e8-07cc-4657-b0d4-6ecad56e0f2a admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:54.174255 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758d8e8-07cc-4657-b0d4-6ecad56e0f2a admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:54.176044 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be70e0c-8a41-47a3-844e-711ff3e5fce6 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:54.179619 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e7032-e195-4d7f-b7a8-0e5332a2004f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:54.180251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e7032-e195-4d7f-b7a8-0e5332a2004f admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:54.180534 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e7032-e195-4d7f-b7a8-0e5332a2004f admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:54.180804 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e7032-e195-4d7f-b7a8-0e5332a2004f admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:54.181496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e7032-e195-4d7f-b7a8-0e5332a2004f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:54.181861 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e261e-a126-4761-a705-c6e320a9c935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:54.182609 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e261e-a126-4761-a705-c6e320a9c935 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:54.182975 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e261e-a126-4761-a705-c6e320a9c935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:54.183122 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e7032-e195-4d7f-b7a8-0e5332a2004f admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:54.183352 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e261e-a126-4761-a705-c6e320a9c935 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:54.184255 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e261e-a126-4761-a705-c6e320a9c935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:54.186275 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e261e-a126-4761-a705-c6e320a9c935 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:54.199402 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1037/2041] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:54.199402 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e7032-e195-4d7f-b7a8-0e5332a2004f admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:54.199855 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1005/2042] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:54.200991 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e261e-a126-4761-a705-c6e320a9c935 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:54.209304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e7032-e195-4d7f-b7a8-0e5332a2004f admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:54.211806 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e261e-a126-4761-a705-c6e320a9c935 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:54.213990 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a9613-9384-472d-b0ab-ff3e9c5b00c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:54.214576 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a9613-9384-472d-b0ab-ff3e9c5b00c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:54.214961 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a9613-9384-472d-b0ab-ff3e9c5b00c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:54.215236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a9613-9384-472d-b0ab-ff3e9c5b00c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:54.215931 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a9613-9384-472d-b0ab-ff3e9c5b00c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:54.216699 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b572be9f-8886-4aa7-af3e-c8384975b056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:54.217353 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572be9f-8886-4aa7-af3e-c8384975b056 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:54.217628 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572be9f-8886-4aa7-af3e-c8384975b056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:54.217741 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a9613-9384-472d-b0ab-ff3e9c5b00c1 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:54.217952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572be9f-8886-4aa7-af3e-c8384975b056 admin admin] PATH_INFO: `/v3/projects/e8e8bb26775044aea7ee69c764dccf15/users/c529b3d9b76549cb90eac41b3cfec6a9/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:54.228555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1038/2043] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:54.231930 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a9613-9384-472d-b0ab-ff3e9c5b00c1 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:54.236116 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b572be9f-8886-4aa7-af3e-c8384975b056 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8e8bb26775044aea7ee69c764dccf15, user_id=c529b3d9b76549cb90eac41b3cfec6a9, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:54.238771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b572be9f-8886-4aa7-af3e-c8384975b056 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:54.243133 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a9613-9384-472d-b0ab-ff3e9c5b00c1 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:54.248612 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd148d3-b2d9-4ffb-8d3b-15d1e9753cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:54.249471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd148d3-b2d9-4ffb-8d3b-15d1e9753cd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:54.249783 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd148d3-b2d9-4ffb-8d3b-15d1e9753cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:54.249970 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:54.250095 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd148d3-b2d9-4ffb-8d3b-15d1e9753cd5 admin admin] PATH_INFO: `/v3/projects/02d1a0bea0c64c28b8ace5e701e2d37d/users/bb9dda90faf94bc2b8aa2e17d1286803/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:54.256470 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1006/2044] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:54 2022] PUT /identity/v3/projects/e8e8bb26775044aea7ee69c764dccf15/users/c529b3d9b76549cb90eac41b3cfec6a9/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:19:54.257969 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b572be9f-8886-4aa7-af3e-c8384975b056 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:54.271194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd148d3-b2d9-4ffb-8d3b-15d1e9753cd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02d1a0bea0c64c28b8ace5e701e2d37d, user_id=bb9dda90faf94bc2b8aa2e17d1286803, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:54.274446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd148d3-b2d9-4ffb-8d3b-15d1e9753cd5 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:54.289277 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1039/2045] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:54 2022] PUT /identity/v3/projects/02d1a0bea0c64c28b8ace5e701e2d37d/users/bb9dda90faf94bc2b8aa2e17d1286803/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:54.291560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd148d3-b2d9-4ffb-8d3b-15d1e9753cd5 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:54.313734 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:54.344124 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b572be9f-8886-4aa7-af3e-c8384975b056 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:54.381115 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd148d3-b2d9-4ffb-8d3b-15d1e9753cd5 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:54.387746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba90a2-bcf3-4275-b2ae-9c0d5e9653c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:54.388457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba90a2-bcf3-4275-b2ae-9c0d5e9653c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:54.388805 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba90a2-bcf3-4275-b2ae-9c0d5e9653c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:54.389097 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba90a2-bcf3-4275-b2ae-9c0d5e9653c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:54.389748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba90a2-bcf3-4275-b2ae-9c0d5e9653c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:54.391983 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba90a2-bcf3-4275-b2ae-9c0d5e9653c0 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:54.402823 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1040/2046] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:54.405042 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243ba63-4f78-4532-af8c-21fd32f039d0 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:54.405342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243ba63-4f78-4532-af8c-21fd32f039d0 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:54.405801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243ba63-4f78-4532-af8c-21fd32f039d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:54.424329 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd19ce-bcd0-47d7-889a-0ebf160c2f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:54.425119 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd19ce-bcd0-47d7-889a-0ebf160c2f7f admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:54.425493 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd19ce-bcd0-47d7-889a-0ebf160c2f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:54.425872 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd19ce-bcd0-47d7-889a-0ebf160c2f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:54.426617 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd19ce-bcd0-47d7-889a-0ebf160c2f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:54.428370 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd19ce-bcd0-47d7-889a-0ebf160c2f7f admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:54.437640 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-f243ba63-4f78-4532-af8c-21fd32f039d0 None None] MFA Rules not processed for user `bb9dda90faf94bc2b8aa2e17d1286803`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:54.439222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f243ba63-4f78-4532-af8c-21fd32f039d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:54.439606 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1007/2047] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:54.445394 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd19ce-bcd0-47d7-889a-0ebf160c2f7f admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:54.455988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd19ce-bcd0-47d7-889a-0ebf160c2f7f admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:54.465148 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f7d16-2610-4b11-af1a-c1cea16216fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:54.466097 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f7d16-2610-4b11-af1a-c1cea16216fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:54.466525 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f7d16-2610-4b11-af1a-c1cea16216fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:54.466877 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f7d16-2610-4b11-af1a-c1cea16216fa admin admin] PATH_INFO: `/v3/projects/e8e8bb26775044aea7ee69c764dccf15/users/c529b3d9b76549cb90eac41b3cfec6a9/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:54.471147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4f7d16-2610-4b11-af1a-c1cea16216fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8e8bb26775044aea7ee69c764dccf15, user_id=c529b3d9b76549cb90eac41b3cfec6a9, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:54.474370 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4f7d16-2610-4b11-af1a-c1cea16216fa admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:54.490757 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1008/2048] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:54 2022] PUT /identity/v3/projects/e8e8bb26775044aea7ee69c764dccf15/users/c529b3d9b76549cb90eac41b3cfec6a9/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:54.492657 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f7d16-2610-4b11-af1a-c1cea16216fa admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:54.526384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1041/2049] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:54.527504 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d8e08-1d7e-40bd-9cda-cf40c7e2fa71 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:54.527830 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d8e08-1d7e-40bd-9cda-cf40c7e2fa71 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:54.528105 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d8e08-1d7e-40bd-9cda-cf40c7e2fa71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:54.565964 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-955d8e08-1d7e-40bd-9cda-cf40c7e2fa71 None None] MFA Rules not processed for user `c529b3d9b76549cb90eac41b3cfec6a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:54.567207 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-955d8e08-1d7e-40bd-9cda-cf40c7e2fa71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:54.587900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f7d16-2610-4b11-af1a-c1cea16216fa admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:54.593203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2aa1a-690d-4528-b08c-b55bb66c90f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:54.593877 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2aa1a-690d-4528-b08c-b55bb66c90f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:54.594150 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2aa1a-690d-4528-b08c-b55bb66c90f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:54.594984 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2aa1a-690d-4528-b08c-b55bb66c90f2 admin admin] PATH_INFO: `/v3/projects/ae0e87fd214d43eea687f00ddfc9d97c/users/e2ff5514ceba4949bbadf499927954cf/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:54.612550 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb2aa1a-690d-4528-b08c-b55bb66c90f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae0e87fd214d43eea687f00ddfc9d97c, user_id=e2ff5514ceba4949bbadf499927954cf, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:54.615308 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb2aa1a-690d-4528-b08c-b55bb66c90f2 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:54.630143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1009/2050] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:54 2022] PUT /identity/v3/projects/ae0e87fd214d43eea687f00ddfc9d97c/users/e2ff5514ceba4949bbadf499927954cf/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:54.633009 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2aa1a-690d-4528-b08c-b55bb66c90f2 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:54.635894 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-1fb2aa1a-690d-4528-b08c-b55bb66c90f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:54.656507 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:54.658486 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1042/2051] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:54.659842 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42690477-de88-46f4-b702-e26d3ae889c3 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:54.660227 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42690477-de88-46f4-b702-e26d3ae889c3 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:54.660572 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42690477-de88-46f4-b702-e26d3ae889c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:54.695105 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-42690477-de88-46f4-b702-e26d3ae889c3 None None] MFA Rules not processed for user `e2ff5514ceba4949bbadf499927954cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:54.696422 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-42690477-de88-46f4-b702-e26d3ae889c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:54.712282 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:54.728753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2aa1a-690d-4528-b08c-b55bb66c90f2 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:54.733237 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50e1b4-08b1-4ec7-a6e2-881fd4653c59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:54.734157 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50e1b4-08b1-4ec7-a6e2-881fd4653c59 service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:54.734600 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50e1b4-08b1-4ec7-a6e2-881fd4653c59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:54.735050 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50e1b4-08b1-4ec7-a6e2-881fd4653c59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:54.773659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1043/2052] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:54.774876 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42690477-de88-46f4-b702-e26d3ae889c3 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:54.776658 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-42690477-de88-46f4-b702-e26d3ae889c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:54.791314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42690477-de88-46f4-b702-e26d3ae889c3 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:54.795321 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ba39c-5d19-481b-bead-0ad5cad68a3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:54.795986 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ba39c-5d19-481b-bead-0ad5cad68a3a service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:54.796267 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ba39c-5d19-481b-bead-0ad5cad68a3a service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:54.796555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ba39c-5d19-481b-bead-0ad5cad68a3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:54.815287 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f50e1b4-08b1-4ec7-a6e2-881fd4653c59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:54.816910 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f50e1b4-08b1-4ec7-a6e2-881fd4653c59 service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:54.825988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1010/2053] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:54.826863 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50e1b4-08b1-4ec7-a6e2-881fd4653c59 service cinder] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:54.828202 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-1f50e1b4-08b1-4ec7-a6e2-881fd4653c59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:54.866329 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221ba39c-5d19-481b-bead-0ad5cad68a3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:54.870988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221ba39c-5d19-481b-bead-0ad5cad68a3a service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:54.880850 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1044/2054] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:54.882270 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ba39c-5d19-481b-bead-0ad5cad68a3a service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:54.939283 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50e1b4-08b1-4ec7-a6e2-881fd4653c59 service cinder] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:54.948454 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ea3ad-2077-4564-9c07-40ac11368c0d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed75c0be809d4a0eb0e2ccc59805b1ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:54.950592 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ea3ad-2077-4564-9c07-40ac11368c0d service glance] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:54.950866 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ea3ad-2077-4564-9c07-40ac11368c0d service glance] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:54.951152 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ea3ad-2077-4564-9c07-40ac11368c0d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:54.967920 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4ea3ad-2077-4564-9c07-40ac11368c0d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:54.970921 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4ea3ad-2077-4564-9c07-40ac11368c0d service glance] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:54.984556 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1011/2055] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:55.010749 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ba39c-5d19-481b-bead-0ad5cad68a3a service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:55.022560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc40dc-dc8d-4b06-b340-63b79d9e8ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:55.023980 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc40dc-dc8d-4b06-b340-63b79d9e8ba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:55.024969 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc40dc-dc8d-4b06-b340-63b79d9e8ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:55.025567 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc40dc-dc8d-4b06-b340-63b79d9e8ba5 admin admin] PATH_INFO: `/v3/users/bb9dda90faf94bc2b8aa2e17d1286803` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:55.028318 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc40dc-dc8d-4b06-b340-63b79d9e8ba5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb9dda90faf94bc2b8aa2e17d1286803)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:55.032113 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc40dc-dc8d-4b06-b340-63b79d9e8ba5 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:55.114209 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0ecc40dc-dc8d-4b06-b340-63b79d9e8ba5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb9dda90faf94bc2b8aa2e17d1286803'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:55.124062 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0ecc40dc-dc8d-4b06-b340-63b79d9e8ba5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb9dda90faf94bc2b8aa2e17d1286803'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:55.125155 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0ecc40dc-dc8d-4b06-b340-63b79d9e8ba5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb9dda90faf94bc2b8aa2e17d1286803'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:55.126344 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0ecc40dc-dc8d-4b06-b340-63b79d9e8ba5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb9dda90faf94bc2b8aa2e17d1286803'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:55.127634 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:55.144012 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0ecc40dc-dc8d-4b06-b340-63b79d9e8ba5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb9dda90faf94bc2b8aa2e17d1286803'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:55.161944 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1045/2056] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:54 2022] DELETE /identity/v3/users/bb9dda90faf94bc2b8aa2e17d1286803 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:55.167268 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af42c29-5bc6-45a8-a694-b6ceb87e92f8 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:55.167896 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af42c29-5bc6-45a8-a694-b6ceb87e92f8 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:55.168177 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af42c29-5bc6-45a8-a694-b6ceb87e92f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:55.196078 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-0af42c29-5bc6-45a8-a694-b6ceb87e92f8 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:55.197948 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-0af42c29-5bc6-45a8-a694-b6ceb87e92f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:55.259726 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:55.294698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1012/2057] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:55.314409 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc40dc-dc8d-4b06-b340-63b79d9e8ba5 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:55.318461 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-0ecc40dc-dc8d-4b06-b340-63b79d9e8ba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:55.331500 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af42c29-5bc6-45a8-a694-b6ceb87e92f8 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:55.337111 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-0af42c29-5bc6-45a8-a694-b6ceb87e92f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:55.358862 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc40dc-dc8d-4b06-b340-63b79d9e8ba5 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:55.369450 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672a790-1167-4476-bc5a-4147992b565d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:55.370276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672a790-1167-4476-bc5a-4147992b565d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:55.370664 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672a790-1167-4476-bc5a-4147992b565d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:55.371154 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672a790-1167-4476-bc5a-4147992b565d admin admin] PATH_INFO: `/v3/users/5510817494fe4f2cb546a0102c364f1f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:55.373482 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672a790-1167-4476-bc5a-4147992b565d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5510817494fe4f2cb546a0102c364f1f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:55.377514 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672a790-1167-4476-bc5a-4147992b565d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:55.467082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a672a790-1167-4476-bc5a-4147992b565d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5510817494fe4f2cb546a0102c364f1f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:55.477122 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a672a790-1167-4476-bc5a-4147992b565d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5510817494fe4f2cb546a0102c364f1f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:55.477875 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a672a790-1167-4476-bc5a-4147992b565d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5510817494fe4f2cb546a0102c364f1f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:55.478515 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a672a790-1167-4476-bc5a-4147992b565d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5510817494fe4f2cb546a0102c364f1f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:55.494537 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a672a790-1167-4476-bc5a-4147992b565d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5510817494fe4f2cb546a0102c364f1f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:55.513322 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af42c29-5bc6-45a8-a694-b6ceb87e92f8 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:55.517072 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1046/2058] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:55 2022] DELETE /identity/v3/users/5510817494fe4f2cb546a0102c364f1f => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:55.518549 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21af09b-da4d-4276-8e2e-5fc9331dce6c None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:55.518917 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21af09b-da4d-4276-8e2e-5fc9331dce6c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:55.519263 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21af09b-da4d-4276-8e2e-5fc9331dce6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:55.544800 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-e21af09b-da4d-4276-8e2e-5fc9331dce6c None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:55.546624 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e21af09b-da4d-4276-8e2e-5fc9331dce6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:55.579011 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9244a-f59b-4b8a-b89f-108871ec3b22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:55.579919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9244a-f59b-4b8a-b89f-108871ec3b22 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:55.580235 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9244a-f59b-4b8a-b89f-108871ec3b22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:55.582317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9244a-f59b-4b8a-b89f-108871ec3b22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:55.585159 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-a4a9244a-f59b-4b8a-b89f-108871ec3b22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:55.633635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1047/2059] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:19:55.642731 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21af09b-da4d-4276-8e2e-5fc9331dce6c None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:55.644606 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e21af09b-da4d-4276-8e2e-5fc9331dce6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:55.659888 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21af09b-da4d-4276-8e2e-5fc9331dce6c None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:55.666572 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71cddf-653c-4947-89d8-4cd874a57d96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:55.667284 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71cddf-653c-4947-89d8-4cd874a57d96 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:55.667831 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71cddf-653c-4947-89d8-4cd874a57d96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:55.668339 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71cddf-653c-4947-89d8-4cd874a57d96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:55.676665 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a9244a-f59b-4b8a-b89f-108871ec3b22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:55.678309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a9244a-f59b-4b8a-b89f-108871ec3b22 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:55.679841 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd71cddf-653c-4947-89d8-4cd874a57d96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:55.681274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd71cddf-653c-4947-89d8-4cd874a57d96 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:55.691537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1013/2060] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:55.692539 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1048/2061] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:19:56.132731 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9244a-f59b-4b8a-b89f-108871ec3b22 service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:56.134204 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-a4a9244a-f59b-4b8a-b89f-108871ec3b22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:56.153555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71cddf-653c-4947-89d8-4cd874a57d96 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:56.155065 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9244a-f59b-4b8a-b89f-108871ec3b22 service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:56.158041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-dd71cddf-653c-4947-89d8-4cd874a57d96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:56.162925 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd28347-6911-49ee-a49c-bc1a7b6465cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:56.164000 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd28347-6911-49ee-a49c-bc1a7b6465cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:56.164361 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd28347-6911-49ee-a49c-bc1a7b6465cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:56.164753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd28347-6911-49ee-a49c-bc1a7b6465cd admin admin] PATH_INFO: `/v3/projects/02d1a0bea0c64c28b8ace5e701e2d37d` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:56.166574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd28347-6911-49ee-a49c-bc1a7b6465cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=02d1a0bea0c64c28b8ace5e701e2d37d)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:56.169118 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd28347-6911-49ee-a49c-bc1a7b6465cd admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:56.184448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71cddf-653c-4947-89d8-4cd874a57d96 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:56.193685 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c2648a-deb7-490a-9126-a666ea219c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:56.194699 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2648a-deb7-490a-9126-a666ea219c85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:56.195251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2648a-deb7-490a-9126-a666ea219c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:56.195636 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2648a-deb7-490a-9126-a666ea219c85 admin admin] PATH_INFO: `/v3/projects/5523d6c106da44dd8dec059df60f2f18` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:56.197980 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c2648a-deb7-490a-9126-a666ea219c85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5523d6c106da44dd8dec059df60f2f18)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:56.201987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c2648a-deb7-490a-9126-a666ea219c85 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:56.235112 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-6bd28347-6911-49ee-a49c-bc1a7b6465cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02d1a0bea0c64c28b8ace5e701e2d37d', 'request_id': 'req-6bd28347-6911-49ee-a49c-bc1a7b6465cd'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:56.238241 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:56.248358 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-6bd28347-6911-49ee-a49c-bc1a7b6465cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02d1a0bea0c64c28b8ace5e701e2d37d', 'request_id': 'req-6bd28347-6911-49ee-a49c-bc1a7b6465cd'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:56.261060 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-6bd28347-6911-49ee-a49c-bc1a7b6465cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02d1a0bea0c64c28b8ace5e701e2d37d', 'request_id': 'req-6bd28347-6911-49ee-a49c-bc1a7b6465cd'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:56.263566 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:56.275939 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-6bd28347-6911-49ee-a49c-bc1a7b6465cd admin admin] The token cache is being invalidate because project 02d1a0bea0c64c28b8ace5e701e2d37d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:56.276400 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-6bd28347-6911-49ee-a49c-bc1a7b6465cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:56.277486 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-6bd28347-6911-49ee-a49c-bc1a7b6465cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:56.279723 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1014/2062] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:56 2022] DELETE /identity/v3/projects/02d1a0bea0c64c28b8ace5e701e2d37d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:56.284568 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c80a2f-1ad7-45a3-87b5-75d71580e08c None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:56.284857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c80a2f-1ad7-45a3-87b5-75d71580e08c None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:56.285162 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c80a2f-1ad7-45a3-87b5-75d71580e08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:56.292545 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a8c2648a-deb7-490a-9126-a666ea219c85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5523d6c106da44dd8dec059df60f2f18', 'request_id': 'req-a8c2648a-deb7-490a-9126-a666ea219c85'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:56.301484 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a8c2648a-deb7-490a-9126-a666ea219c85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5523d6c106da44dd8dec059df60f2f18', 'request_id': 'req-a8c2648a-deb7-490a-9126-a666ea219c85'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:56.306322 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-64c80a2f-1ad7-45a3-87b5-75d71580e08c None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:56.307568 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-64c80a2f-1ad7-45a3-87b5-75d71580e08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:56.309376 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a8c2648a-deb7-490a-9126-a666ea219c85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5523d6c106da44dd8dec059df60f2f18', 'request_id': 'req-a8c2648a-deb7-490a-9126-a666ea219c85'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:56.328062 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a8c2648a-deb7-490a-9126-a666ea219c85 admin admin] The token cache is being invalidate because project 5523d6c106da44dd8dec059df60f2f18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:56.328338 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a8c2648a-deb7-490a-9126-a666ea219c85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:56.329714 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a8c2648a-deb7-490a-9126-a666ea219c85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:56.331260 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1049/2063] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:56 2022] DELETE /identity/v3/projects/5523d6c106da44dd8dec059df60f2f18 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:56.336312 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c2648a-deb7-490a-9126-a666ea219c85 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:56.338440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a8c2648a-deb7-490a-9126-a666ea219c85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:56.394265 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1015/2064] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:19:56.403976 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09f6cf-5b47-41c4-a71d-25e7650f6814 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:56.404277 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09f6cf-5b47-41c4-a71d-25e7650f6814 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:56.404726 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09f6cf-5b47-41c4-a71d-25e7650f6814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:56.442376 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c2648a-deb7-490a-9126-a666ea219c85 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:56.447903 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504dbf95-f634-4085-a592-0689da936941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:56.448569 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504dbf95-f634-4085-a592-0689da936941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:56.448849 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504dbf95-f634-4085-a592-0689da936941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:56.449127 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504dbf95-f634-4085-a592-0689da936941 admin admin] PATH_INFO: `/v3/users/de1b2d01d632470a9aadbe0fb2a30e19` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:56.450463 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504dbf95-f634-4085-a592-0689da936941 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de1b2d01d632470a9aadbe0fb2a30e19)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:56.453433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504dbf95-f634-4085-a592-0689da936941 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:56.455315 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-aa09f6cf-5b47-41c4-a71d-25e7650f6814 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:56.456895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-aa09f6cf-5b47-41c4-a71d-25e7650f6814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:56.474420 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1016/2065] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:56.480417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09f6cf-5b47-41c4-a71d-25e7650f6814 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:56.500294 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09f6cf-5b47-41c4-a71d-25e7650f6814 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:56.506428 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f873da5-caf9-49ef-8a18-ebbb0145fcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:56.507142 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f873da5-caf9-49ef-8a18-ebbb0145fcb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:56.507673 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f873da5-caf9-49ef-8a18-ebbb0145fcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:56.507959 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f873da5-caf9-49ef-8a18-ebbb0145fcb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:56.508680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f873da5-caf9-49ef-8a18-ebbb0145fcb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:19:56.508991 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f873da5-caf9-49ef-8a18-ebbb0145fcb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:56.514674 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f873da5-caf9-49ef-8a18-ebbb0145fcb7 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:56.546103 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1017/2066] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:19:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:19:56.549061 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-504dbf95-f634-4085-a592-0689da936941 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de1b2d01d632470a9aadbe0fb2a30e19'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:56.551111 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f873da5-caf9-49ef-8a18-ebbb0145fcb7 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:56.560501 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-504dbf95-f634-4085-a592-0689da936941 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de1b2d01d632470a9aadbe0fb2a30e19'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:56.561311 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-504dbf95-f634-4085-a592-0689da936941 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de1b2d01d632470a9aadbe0fb2a30e19'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:56.561943 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-504dbf95-f634-4085-a592-0689da936941 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de1b2d01d632470a9aadbe0fb2a30e19'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:56.579375 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f873da5-caf9-49ef-8a18-ebbb0145fcb7 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:56.580702 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-504dbf95-f634-4085-a592-0689da936941 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de1b2d01d632470a9aadbe0fb2a30e19'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:56.586859 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb6061-90b1-4f32-b494-aa6002d7c57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:56.588513 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb6061-90b1-4f32-b494-aa6002d7c57a admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:56.589418 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb6061-90b1-4f32-b494-aa6002d7c57a admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:56.589941 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb6061-90b1-4f32-b494-aa6002d7c57a admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:56.590803 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb6061-90b1-4f32-b494-aa6002d7c57a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:56.593314 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb6061-90b1-4f32-b494-aa6002d7c57a admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:56.601002 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1050/2067] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:56 2022] DELETE /identity/v3/users/de1b2d01d632470a9aadbe0fb2a30e19 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:56.657330 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1018/2068] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:56 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:56.661546 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504dbf95-f634-4085-a592-0689da936941 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:56.663243 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-504dbf95-f634-4085-a592-0689da936941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:56.715481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:56.777496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504dbf95-f634-4085-a592-0689da936941 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:56.843559 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a47f36-6ee4-46ec-ab30-9bb865e6fbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:56.844174 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a47f36-6ee4-46ec-ab30-9bb865e6fbb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:56.844496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a47f36-6ee4-46ec-ab30-9bb865e6fbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:56.844771 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a47f36-6ee4-46ec-ab30-9bb865e6fbb9 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:56.845409 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a47f36-6ee4-46ec-ab30-9bb865e6fbb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:56.847327 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a47f36-6ee4-46ec-ab30-9bb865e6fbb9 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:56.887182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1051/2069] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:56 2022] POST /identity/v3/users => generated 289 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:19:56.891468 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb6061-90b1-4f32-b494-aa6002d7c57a admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:56.902413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb6061-90b1-4f32-b494-aa6002d7c57a admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:56.907157 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88370d0e-b819-42e9-a758-1ca2e0d1e4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:56.907957 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88370d0e-b819-42e9-a758-1ca2e0d1e4ba admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:56.908238 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88370d0e-b819-42e9-a758-1ca2e0d1e4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:56.908513 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88370d0e-b819-42e9-a758-1ca2e0d1e4ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:56.909183 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88370d0e-b819-42e9-a758-1ca2e0d1e4ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:56.914534 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88370d0e-b819-42e9-a758-1ca2e0d1e4ba admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:56.923807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1019/2070] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:19:56.928072 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a47f36-6ee4-46ec-ab30-9bb865e6fbb9 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:56.939807 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a47f36-6ee4-46ec-ab30-9bb865e6fbb9 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:56.945126 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0641a64-c772-456e-8c06-041a63bdb2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:56.945847 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0641a64-c772-456e-8c06-041a63bdb2eb admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:56.946374 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0641a64-c772-456e-8c06-041a63bdb2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:56.946714 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0641a64-c772-456e-8c06-041a63bdb2eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:56.947443 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0641a64-c772-456e-8c06-041a63bdb2eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:56.949598 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0641a64-c772-456e-8c06-041a63bdb2eb admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:56.960404 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1052/2071] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:56.964562 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88370d0e-b819-42e9-a758-1ca2e0d1e4ba admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:56.976068 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88370d0e-b819-42e9-a758-1ca2e0d1e4ba admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:56.982795 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced088e-2256-4aad-aa8c-b56eb85a3700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:56.983759 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced088e-2256-4aad-aa8c-b56eb85a3700 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:56.984157 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced088e-2256-4aad-aa8c-b56eb85a3700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:56.984439 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced088e-2256-4aad-aa8c-b56eb85a3700 admin admin] PATH_INFO: `/v3/projects/296bd099e9f8492aaeae70cf38f969c9/users/d26f8c676bc549eeb2a3b54cd577f95c/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:57.007073 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced088e-2256-4aad-aa8c-b56eb85a3700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=296bd099e9f8492aaeae70cf38f969c9, user_id=d26f8c676bc549eeb2a3b54cd577f95c, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:57.011583 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced088e-2256-4aad-aa8c-b56eb85a3700 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:57.032434 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1020/2072] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:56 2022] PUT /identity/v3/projects/296bd099e9f8492aaeae70cf38f969c9/users/d26f8c676bc549eeb2a3b54cd577f95c/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:57.042324 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4e4b6-efed-4a59-b98e-3d6245aeabe8 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:57.042615 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4e4b6-efed-4a59-b98e-3d6245aeabe8 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:57.042889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4e4b6-efed-4a59-b98e-3d6245aeabe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:57.076980 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-0fc4e4b6-efed-4a59-b98e-3d6245aeabe8 None None] MFA Rules not processed for user `d26f8c676bc549eeb2a3b54cd577f95c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:57.078596 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-0fc4e4b6-efed-4a59-b98e-3d6245aeabe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:57.092661 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced088e-2256-4aad-aa8c-b56eb85a3700 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:57.096475 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-9ced088e-2256-4aad-aa8c-b56eb85a3700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:57.173470 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1053/2073] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:57.185589 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc4e4b6-efed-4a59-b98e-3d6245aeabe8 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:57.187033 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-0fc4e4b6-efed-4a59-b98e-3d6245aeabe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:57.204701 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced088e-2256-4aad-aa8c-b56eb85a3700 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:57.209710 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a3386b-b13a-4abd-aad6-72fc2af1173a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:57.210543 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3386b-b13a-4abd-aad6-72fc2af1173a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:57.210720 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3386b-b13a-4abd-aad6-72fc2af1173a admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:57.210927 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3386b-b13a-4abd-aad6-72fc2af1173a admin admin] PATH_INFO: `/v3/projects/6fdf284bc2504b20a28bd9b21d2564a4` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:57.212694 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a3386b-b13a-4abd-aad6-72fc2af1173a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fdf284bc2504b20a28bd9b21d2564a4)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:57.214690 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a3386b-b13a-4abd-aad6-72fc2af1173a admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:57.223324 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:57.273275 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc4e4b6-efed-4a59-b98e-3d6245aeabe8 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:57.280406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:57.286653 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b4a3386b-b13a-4abd-aad6-72fc2af1173a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fdf284bc2504b20a28bd9b21d2564a4', 'request_id': 'req-b4a3386b-b13a-4abd-aad6-72fc2af1173a'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:57.295622 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b4a3386b-b13a-4abd-aad6-72fc2af1173a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fdf284bc2504b20a28bd9b21d2564a4', 'request_id': 'req-b4a3386b-b13a-4abd-aad6-72fc2af1173a'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:57.302534 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b4a3386b-b13a-4abd-aad6-72fc2af1173a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fdf284bc2504b20a28bd9b21d2564a4', 'request_id': 'req-b4a3386b-b13a-4abd-aad6-72fc2af1173a'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:57.314236 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b4a3386b-b13a-4abd-aad6-72fc2af1173a admin admin] The token cache is being invalidate because project 6fdf284bc2504b20a28bd9b21d2564a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:57.314511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b4a3386b-b13a-4abd-aad6-72fc2af1173a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:57.315180 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b4a3386b-b13a-4abd-aad6-72fc2af1173a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:57.316780 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1021/2074] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:57 2022] DELETE /identity/v3/projects/6fdf284bc2504b20a28bd9b21d2564a4 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:57.322511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6ebe6-18d6-40f2-8cd3-0f919aa2c7aa None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:57.322797 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6ebe6-18d6-40f2-8cd3-0f919aa2c7aa None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:57.323100 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6ebe6-18d6-40f2-8cd3-0f919aa2c7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:57.331683 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f0d5c-1be9-479c-99fb-06c1d17fbb40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:57.332321 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f0d5c-1be9-479c-99fb-06c1d17fbb40 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:57.332597 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f0d5c-1be9-479c-99fb-06c1d17fbb40 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:57.332877 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f0d5c-1be9-479c-99fb-06c1d17fbb40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:57.335448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-647f0d5c-1be9-479c-99fb-06c1d17fbb40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:57.346386 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-3ef6ebe6-18d6-40f2-8cd3-0f919aa2c7aa None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:57.347624 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-3ef6ebe6-18d6-40f2-8cd3-0f919aa2c7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:57.410880 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f0d5c-1be9-479c-99fb-06c1d17fbb40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:57.412385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f0d5c-1be9-479c-99fb-06c1d17fbb40 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:57.422023 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1054/2075] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:19:57.433841 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1022/2076] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:19:57.444600 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ceeaf-d7f6-423a-bfbd-1b3c5918dd90 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:57.445873 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ceeaf-d7f6-423a-bfbd-1b3c5918dd90 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:57.446370 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ceeaf-d7f6-423a-bfbd-1b3c5918dd90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:57.476957 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-8a4ceeaf-d7f6-423a-bfbd-1b3c5918dd90 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:57.478447 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8a4ceeaf-d7f6-423a-bfbd-1b3c5918dd90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:57.487946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1055/2077] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:57.492493 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef6ebe6-18d6-40f2-8cd3-0f919aa2c7aa None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:57.505201 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef6ebe6-18d6-40f2-8cd3-0f919aa2c7aa None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:57.511474 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de33e3-2587-47a3-a630-9210e1088479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:57.512179 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de33e3-2587-47a3-a630-9210e1088479 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:57.512571 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de33e3-2587-47a3-a630-9210e1088479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:57.512875 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de33e3-2587-47a3-a630-9210e1088479 admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:57.513544 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de33e3-2587-47a3-a630-9210e1088479 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:19:57.513850 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de33e3-2587-47a3-a630-9210e1088479 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:57.515492 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de33e3-2587-47a3-a630-9210e1088479 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:57.529304 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1023/2078] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:19:57.535189 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ceeaf-d7f6-423a-bfbd-1b3c5918dd90 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:57.544775 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ceeaf-d7f6-423a-bfbd-1b3c5918dd90 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:57.550235 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc687f4a-8be6-4087-a928-22fdfcf40dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:57.550949 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc687f4a-8be6-4087-a928-22fdfcf40dc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:57.551284 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc687f4a-8be6-4087-a928-22fdfcf40dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:57.551654 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc687f4a-8be6-4087-a928-22fdfcf40dc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:57.552465 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc687f4a-8be6-4087-a928-22fdfcf40dc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:57.554489 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc687f4a-8be6-4087-a928-22fdfcf40dc7 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:57.595382 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1056/2079] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:57 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:57.597894 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de33e3-2587-47a3-a630-9210e1088479 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:57.610537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de33e3-2587-47a3-a630-9210e1088479 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:57.687007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd87e11-a754-4abc-8d7b-baaef737b8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:57.687560 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd87e11-a754-4abc-8d7b-baaef737b8c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:57.687923 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd87e11-a754-4abc-8d7b-baaef737b8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:57.688296 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd87e11-a754-4abc-8d7b-baaef737b8c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:57.689007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd87e11-a754-4abc-8d7b-baaef737b8c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:57.690942 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd87e11-a754-4abc-8d7b-baaef737b8c3 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:57.717300 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1024/2080] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:57 2022] POST /identity/v3/users => generated 287 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:57.729302 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc687f4a-8be6-4087-a928-22fdfcf40dc7 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:57.756377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc687f4a-8be6-4087-a928-22fdfcf40dc7 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:57.766908 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b78248-4a34-48a3-a113-35e587e64d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:57.771063 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b78248-4a34-48a3-a113-35e587e64d56 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:57.771461 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b78248-4a34-48a3-a113-35e587e64d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:57.771863 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b78248-4a34-48a3-a113-35e587e64d56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:57.772654 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b78248-4a34-48a3-a113-35e587e64d56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:57.774879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b78248-4a34-48a3-a113-35e587e64d56 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:57.796514 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1057/2081] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:57.807130 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd87e11-a754-4abc-8d7b-baaef737b8c3 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:57.825177 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd87e11-a754-4abc-8d7b-baaef737b8c3 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:57.833339 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43c5cf-c322-4827-8ecf-8eed53017389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:57.834340 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43c5cf-c322-4827-8ecf-8eed53017389 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:57.834875 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43c5cf-c322-4827-8ecf-8eed53017389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:57.835395 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43c5cf-c322-4827-8ecf-8eed53017389 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:57.836347 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43c5cf-c322-4827-8ecf-8eed53017389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:57.838266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43c5cf-c322-4827-8ecf-8eed53017389 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:57.844433 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:57.851823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1025/2082] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:57.862183 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b78248-4a34-48a3-a113-35e587e64d56 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:57.876219 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b78248-4a34-48a3-a113-35e587e64d56 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:57.882386 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97e9a3-1701-4486-b382-6a34f3b4a4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:57.883273 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97e9a3-1701-4486-b382-6a34f3b4a4c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:57.883462 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97e9a3-1701-4486-b382-6a34f3b4a4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:57.883908 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97e9a3-1701-4486-b382-6a34f3b4a4c4 admin admin] PATH_INFO: `/v3/projects/401257a26129402e9c143fc26e8d9e02/users/597b4c17e2ba43b794ff8dbc25d2a86b/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:57.903590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d97e9a3-1701-4486-b382-6a34f3b4a4c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=401257a26129402e9c143fc26e8d9e02, user_id=597b4c17e2ba43b794ff8dbc25d2a86b, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:57.907599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d97e9a3-1701-4486-b382-6a34f3b4a4c4 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:57.941414 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1058/2083] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:57 2022] PUT /identity/v3/projects/401257a26129402e9c143fc26e8d9e02/users/597b4c17e2ba43b794ff8dbc25d2a86b/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:19:57.954592 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ad51e-7d43-4beb-a0b7-65074dc9efdd None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:57.955090 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ad51e-7d43-4beb-a0b7-65074dc9efdd None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:57.955376 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ad51e-7d43-4beb-a0b7-65074dc9efdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:57.987914 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-ff5ad51e-7d43-4beb-a0b7-65074dc9efdd None None] MFA Rules not processed for user `597b4c17e2ba43b794ff8dbc25d2a86b`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:57.989384 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-ff5ad51e-7d43-4beb-a0b7-65074dc9efdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:58.067791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1026/2084] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:58.082158 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97e9a3-1701-4486-b382-6a34f3b4a4c4 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:58.083997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3d97e9a3-1701-4486-b382-6a34f3b4a4c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:58.189570 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97e9a3-1701-4486-b382-6a34f3b4a4c4 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:58.199800 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b54a3-cc2f-4d01-bd36-493a643de934 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:58.200750 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b54a3-cc2f-4d01-bd36-493a643de934 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:58.201228 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b54a3-cc2f-4d01-bd36-493a643de934 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:58.201690 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b54a3-cc2f-4d01-bd36-493a643de934 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:58.212764 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b54a3-cc2f-4d01-bd36-493a643de934 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:58.214745 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b54a3-cc2f-4d01-bd36-493a643de934 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:58.234150 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:58.255215 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1059/2085] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:19:58.280118 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ad51e-7d43-4beb-a0b7-65074dc9efdd None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:58.374812 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ad51e-7d43-4beb-a0b7-65074dc9efdd None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:58.383333 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49063082-2b6e-4cf7-a7bb-ef82e0218762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:58.384085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49063082-2b6e-4cf7-a7bb-ef82e0218762 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:58.384444 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49063082-2b6e-4cf7-a7bb-ef82e0218762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:58.384760 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49063082-2b6e-4cf7-a7bb-ef82e0218762 admin admin] PATH_INFO: `/v3/users/597b4c17e2ba43b794ff8dbc25d2a86b` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:58.386030 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49063082-2b6e-4cf7-a7bb-ef82e0218762 admin admin] RBAC: Authorizing `identity:delete_user(user_id=597b4c17e2ba43b794ff8dbc25d2a86b)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:58.388306 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49063082-2b6e-4cf7-a7bb-ef82e0218762 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:58.442808 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-49063082-2b6e-4cf7-a7bb-ef82e0218762 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '597b4c17e2ba43b794ff8dbc25d2a86b'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:58.451086 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-49063082-2b6e-4cf7-a7bb-ef82e0218762 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '597b4c17e2ba43b794ff8dbc25d2a86b'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:58.451799 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-49063082-2b6e-4cf7-a7bb-ef82e0218762 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '597b4c17e2ba43b794ff8dbc25d2a86b'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:58.464832 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-49063082-2b6e-4cf7-a7bb-ef82e0218762 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '597b4c17e2ba43b794ff8dbc25d2a86b'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:58.472322 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-49063082-2b6e-4cf7-a7bb-ef82e0218762 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '597b4c17e2ba43b794ff8dbc25d2a86b'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:58.479069 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1027/2086] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:19:58 2022] DELETE /identity/v3/users/597b4c17e2ba43b794ff8dbc25d2a86b => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:58.484290 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4d72f-dcda-421f-be62-1656a83ae297 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:58.484574 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4d72f-dcda-421f-be62-1656a83ae297 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:58.484847 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4d72f-dcda-421f-be62-1656a83ae297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:58.505456 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-38b4d72f-dcda-421f-be62-1656a83ae297 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:58.507048 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-38b4d72f-dcda-421f-be62-1656a83ae297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:58.568072 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1060/2087] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:58.580020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49063082-2b6e-4cf7-a7bb-ef82e0218762 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:58.581796 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-49063082-2b6e-4cf7-a7bb-ef82e0218762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:58.593424 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:58.649528 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49063082-2b6e-4cf7-a7bb-ef82e0218762 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:58.653396 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d5ea7-0615-45ec-9141-3e7e43602d2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:58.654043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d5ea7-0615-45ec-9141-3e7e43602d2e service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:58.654278 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d5ea7-0615-45ec-9141-3e7e43602d2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:58.654552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d5ea7-0615-45ec-9141-3e7e43602d2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:58.665454 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7d5ea7-0615-45ec-9141-3e7e43602d2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:58.666898 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7d5ea7-0615-45ec-9141-3e7e43602d2e service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:58.677464 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1028/2088] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:19:58.991670 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4d72f-dcda-421f-be62-1656a83ae297 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:58.993869 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-38b4d72f-dcda-421f-be62-1656a83ae297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:59.015387 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4d72f-dcda-421f-be62-1656a83ae297 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:59.027069 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e2441-4582-47ab-ae67-cc80d4a97fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:59.028541 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e2441-4582-47ab-ae67-cc80d4a97fb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:59.029361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e2441-4582-47ab-ae67-cc80d4a97fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:59.030141 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e2441-4582-47ab-ae67-cc80d4a97fb9 admin admin] PATH_INFO: `/v3/projects/401257a26129402e9c143fc26e8d9e02` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:59.033436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e2441-4582-47ab-ae67-cc80d4a97fb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=401257a26129402e9c143fc26e8d9e02)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:59.038478 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e2441-4582-47ab-ae67-cc80d4a97fb9 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:59.117702 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-474e2441-4582-47ab-ae67-cc80d4a97fb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '401257a26129402e9c143fc26e8d9e02', 'request_id': 'req-474e2441-4582-47ab-ae67-cc80d4a97fb9'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:59.124160 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-474e2441-4582-47ab-ae67-cc80d4a97fb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '401257a26129402e9c143fc26e8d9e02', 'request_id': 'req-474e2441-4582-47ab-ae67-cc80d4a97fb9'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:59.130782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-474e2441-4582-47ab-ae67-cc80d4a97fb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '401257a26129402e9c143fc26e8d9e02', 'request_id': 'req-474e2441-4582-47ab-ae67-cc80d4a97fb9'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:59.145271 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-474e2441-4582-47ab-ae67-cc80d4a97fb9 admin admin] The token cache is being invalidate because project 401257a26129402e9c143fc26e8d9e02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:19:59.145588 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-474e2441-4582-47ab-ae67-cc80d4a97fb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:59.146255 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-474e2441-4582-47ab-ae67-cc80d4a97fb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:19:59.148222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1061/2089] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:19:58 2022] DELETE /identity/v3/projects/401257a26129402e9c143fc26e8d9e02 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:59.155713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51df46-c30f-4fd6-abd2-721a852d9c41 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:59.155932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51df46-c30f-4fd6-abd2-721a852d9c41 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:59.156208 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51df46-c30f-4fd6-abd2-721a852d9c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:59.177026 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-7d51df46-c30f-4fd6-abd2-721a852d9c41 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:59.178282 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-7d51df46-c30f-4fd6-abd2-721a852d9c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:59.252355 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1029/2090] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:59.254470 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e71f8-bde7-4e87-ac2c-8b2c50de8c26 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:59.254760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e71f8-bde7-4e87-ac2c-8b2c50de8c26 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:59.255040 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e71f8-bde7-4e87-ac2c-8b2c50de8c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:59.280118 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-8a8e71f8-bde7-4e87-ac2c-8b2c50de8c26 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:59.280842 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8a8e71f8-bde7-4e87-ac2c-8b2c50de8c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:59.289293 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1062/2091] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:19:59.294396 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d51df46-c30f-4fd6-abd2-721a852d9c41 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:59.305389 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d51df46-c30f-4fd6-abd2-721a852d9c41 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:59.310529 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80b479-b986-4764-a40d-f9dc4ca25535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:59.311149 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80b479-b986-4764-a40d-f9dc4ca25535 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:59.311421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80b479-b986-4764-a40d-f9dc4ca25535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:59.311762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80b479-b986-4764-a40d-f9dc4ca25535 admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:59.312421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd80b479-b986-4764-a40d-f9dc4ca25535 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:19:59.312725 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd80b479-b986-4764-a40d-f9dc4ca25535 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:59.314822 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd80b479-b986-4764-a40d-f9dc4ca25535 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:59.329600 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1030/2092] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:19:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:19:59.334377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e71f8-bde7-4e87-ac2c-8b2c50de8c26 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:59.344622 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e71f8-bde7-4e87-ac2c-8b2c50de8c26 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:59.350055 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a232a8-9cc0-41d4-a00f-ab5cfbc8906a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:59.350055 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a232a8-9cc0-41d4-a00f-ab5cfbc8906a admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:59.350055 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a232a8-9cc0-41d4-a00f-ab5cfbc8906a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:59.350428 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a232a8-9cc0-41d4-a00f-ab5cfbc8906a admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:59.350927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a232a8-9cc0-41d4-a00f-ab5cfbc8906a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:59.352873 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a232a8-9cc0-41d4-a00f-ab5cfbc8906a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:59.389520 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1063/2093] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:19:59 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:59.394841 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80b479-b986-4764-a40d-f9dc4ca25535 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:59.408937 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80b479-b986-4764-a40d-f9dc4ca25535 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:59.474709 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423f223-ffcd-4bd1-b072-78d9fd82807c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:59.475859 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423f223-ffcd-4bd1-b072-78d9fd82807c admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:59.475859 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423f223-ffcd-4bd1-b072-78d9fd82807c admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:59.476085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423f223-ffcd-4bd1-b072-78d9fd82807c admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:59.476657 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f423f223-ffcd-4bd1-b072-78d9fd82807c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:59.478735 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f423f223-ffcd-4bd1-b072-78d9fd82807c admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:59.503559 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1031/2094] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:19:59 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:19:59.506330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a232a8-9cc0-41d4-a00f-ab5cfbc8906a admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:59.519237 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a232a8-9cc0-41d4-a00f-ab5cfbc8906a admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:59.529550 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87bfd0-0757-48d4-9270-a69a124aa0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:59.530292 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87bfd0-0757-48d4-9270-a69a124aa0d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:59.530583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87bfd0-0757-48d4-9270-a69a124aa0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:59.530862 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87bfd0-0757-48d4-9270-a69a124aa0d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:59.531552 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe87bfd0-0757-48d4-9270-a69a124aa0d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:59.533265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe87bfd0-0757-48d4-9270-a69a124aa0d0 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:59.548804 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1064/2095] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:59.553182 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423f223-ffcd-4bd1-b072-78d9fd82807c admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:59.566704 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423f223-ffcd-4bd1-b072-78d9fd82807c admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:59.572479 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be79cf3-729c-464f-a600-5ef834a13b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:59.573147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be79cf3-729c-464f-a600-5ef834a13b8b admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:59.573467 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be79cf3-729c-464f-a600-5ef834a13b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:59.573776 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be79cf3-729c-464f-a600-5ef834a13b8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:59.574544 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be79cf3-729c-464f-a600-5ef834a13b8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:59.576336 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be79cf3-729c-464f-a600-5ef834a13b8b admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:59.588935 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1032/2096] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:19:59.593354 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87bfd0-0757-48d4-9270-a69a124aa0d0 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:59.603734 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87bfd0-0757-48d4-9270-a69a124aa0d0 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:59.608946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a406729-c707-4d89-a7dc-bc155a52a8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:59.610287 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a406729-c707-4d89-a7dc-bc155a52a8ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:59.610580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a406729-c707-4d89-a7dc-bc155a52a8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:59.610855 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a406729-c707-4d89-a7dc-bc155a52a8ce admin admin] PATH_INFO: `/v3/projects/7587d39b1b0143c39b17d29995b4a8dc/users/07b88b4429d74b16921f224a3427d669/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:59.628446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a406729-c707-4d89-a7dc-bc155a52a8ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=7587d39b1b0143c39b17d29995b4a8dc, user_id=07b88b4429d74b16921f224a3427d669, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:59.631156 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a406729-c707-4d89-a7dc-bc155a52a8ce admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:59.646683 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1065/2097] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:19:59 2022] PUT /identity/v3/projects/7587d39b1b0143c39b17d29995b4a8dc/users/07b88b4429d74b16921f224a3427d669/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:19:59.657488 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d81038-845c-4244-b2b8-ea51cb8bdfe5 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:59.658257 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d81038-845c-4244-b2b8-ea51cb8bdfe5 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:59.658257 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d81038-845c-4244-b2b8-ea51cb8bdfe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:59.693362 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-01d81038-845c-4244-b2b8-ea51cb8bdfe5 None None] MFA Rules not processed for user `07b88b4429d74b16921f224a3427d669`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:19:59.694511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-01d81038-845c-4244-b2b8-ea51cb8bdfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:59.700221 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:59.767304 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1033/2098] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:19:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:19:59.779977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a406729-c707-4d89-a7dc-bc155a52a8ce admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:19:59.782129 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5a406729-c707-4d89-a7dc-bc155a52a8ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:19:59.810117 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:19:59.853323 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a406729-c707-4d89-a7dc-bc155a52a8ce admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:19:59.857444 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb44892-1f82-42c1-96af-d7fcb12cadf9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed75c0be809d4a0eb0e2ccc59805b1ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:19:59.858089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb44892-1f82-42c1-96af-d7fcb12cadf9 service glance] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:19:59.858364 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb44892-1f82-42c1-96af-d7fcb12cadf9 service glance] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:19:59.858633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb44892-1f82-42c1-96af-d7fcb12cadf9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:19:59.871616 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb44892-1f82-42c1-96af-d7fcb12cadf9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:19:59.873138 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb44892-1f82-42c1-96af-d7fcb12cadf9 service glance] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:19:59.883818 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1066/2099] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:19:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:20:04.166360 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d81038-845c-4244-b2b8-ea51cb8bdfe5 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:04.168560 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-01d81038-845c-4244-b2b8-ea51cb8bdfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:04.319222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d81038-845c-4244-b2b8-ea51cb8bdfe5 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:04.325850 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff175de-5340-4cbd-86de-f8b088056164 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:04.327749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff175de-5340-4cbd-86de-f8b088056164 service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:04.328288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff175de-5340-4cbd-86de-f8b088056164 service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:04.328827 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff175de-5340-4cbd-86de-f8b088056164 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:04.331149 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-6ff175de-5340-4cbd-86de-f8b088056164 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:04.447881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff175de-5340-4cbd-86de-f8b088056164 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:04.449806 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff175de-5340-4cbd-86de-f8b088056164 service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:04.462104 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1034/2100] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:20:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:20:09.879309 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=17 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:20:14.456417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=74 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:20:50.942012 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e080af-94ff-4eb0-8820-5af92917dc72 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:50.942950 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e080af-94ff-4eb0-8820-5af92917dc72 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:50.942950 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e080af-94ff-4eb0-8820-5af92917dc72 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:50.944084 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1067/2101] 10.222.0.16 () {54 vars in 921 bytes} [Mon Sep 5 12:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:20:50.985352 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc71cb4-271b-4507-93c2-a1e57146ddf3 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:50.987292 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc71cb4-271b-4507-93c2-a1e57146ddf3 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:50.987917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc71cb4-271b-4507-93c2-a1e57146ddf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:51.017718 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-9dc71cb4-271b-4507-93c2-a1e57146ddf3 None None] MFA Rules not processed for user `524bcfffcdfa4790885e0d4cbd66fa80`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:20:51.019135 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-9dc71cb4-271b-4507-93c2-a1e57146ddf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:51.099569 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1035/2102] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Sep 5 12:20:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:20:51.121974 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e56ade-5de6-4420-8e26-07975f97d981 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:51.122603 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e56ade-5de6-4420-8e26-07975f97d981 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:51.123088 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e56ade-5de6-4420-8e26-07975f97d981 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:51.124756 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1068/2103] 10.222.0.16 () {54 vars in 961 bytes} [Mon Sep 5 12:20:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:20:51.131078 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ca282-607c-4dc6-a6e6-b1fcdf57b904 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:51.131370 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ca282-607c-4dc6-a6e6-b1fcdf57b904 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:51.131703 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ca282-607c-4dc6-a6e6-b1fcdf57b904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:51.156635 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-b43ca282-607c-4dc6-a6e6-b1fcdf57b904 None None] MFA Rules not processed for user `524bcfffcdfa4790885e0d4cbd66fa80`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:20:51.158897 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-b43ca282-607c-4dc6-a6e6-b1fcdf57b904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:51.168000 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1036/2104] 10.222.0.16 () {58 vars in 1058 bytes} [Mon Sep 5 12:20:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:20:51.180619 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e56ade-5de6-4420-8e26-07975f97d981 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:51.199743 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e56ade-5de6-4420-8e26-07975f97d981 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:51.206376 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9216ec0-a152-408d-a35e-9c8e1fb6fbda service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524bcfffcdfa4790885e0d4cbd66fa80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:51.207376 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9216ec0-a152-408d-a35e-9c8e1fb6fbda service placement] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:51.208124 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9216ec0-a152-408d-a35e-9c8e1fb6fbda service placement] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:51.208644 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9216ec0-a152-408d-a35e-9c8e1fb6fbda service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:51.223691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9216ec0-a152-408d-a35e-9c8e1fb6fbda service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:51.226857 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9216ec0-a152-408d-a35e-9c8e1fb6fbda service placement] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:51.241516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1069/2105] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:20:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:20:51.727427 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a2018-c9ad-4671-9fe3-49180f140e2f None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:51.727795 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a2018-c9ad-4671-9fe3-49180f140e2f None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:51.728042 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a2018-c9ad-4671-9fe3-49180f140e2f None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:51.729784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1037/2106] 10.222.0.16 () {54 vars in 921 bytes} [Mon Sep 5 12:20:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:20:51.961566 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d096f72-3e34-4919-af91-a3c60baee444 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:51.962125 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d096f72-3e34-4919-af91-a3c60baee444 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:51.962449 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d096f72-3e34-4919-af91-a3c60baee444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:51.998198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-2d096f72-3e34-4919-af91-a3c60baee444 None None] MFA Rules not processed for user `524bcfffcdfa4790885e0d4cbd66fa80`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:20:51.999705 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2d096f72-3e34-4919-af91-a3c60baee444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:52.006204 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a2018-c9ad-4671-9fe3-49180f140e2f None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:52.008300 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-792a2018-c9ad-4671-9fe3-49180f140e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:52.012834 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1070/2107] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Sep 5 12:20:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:20:52.025860 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d096f72-3e34-4919-af91-a3c60baee444 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:52.042757 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d096f72-3e34-4919-af91-a3c60baee444 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:52.049864 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f2fbd-ab1b-4ef3-bf3c-d45608cc5d0d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524bcfffcdfa4790885e0d4cbd66fa80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:52.050779 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f2fbd-ab1b-4ef3-bf3c-d45608cc5d0d service placement] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:52.051058 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f2fbd-ab1b-4ef3-bf3c-d45608cc5d0d service placement] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:52.051510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f2fbd-ab1b-4ef3-bf3c-d45608cc5d0d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:52.061861 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f2fbd-ab1b-4ef3-bf3c-d45608cc5d0d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:52.063894 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f2fbd-ab1b-4ef3-bf3c-d45608cc5d0d service placement] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:52.075677 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1071/2108] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:20:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:20:52.104104 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a2018-c9ad-4671-9fe3-49180f140e2f None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:52.109762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed760c-49bf-4e69-a4c4-cbc6e0346865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:52.110412 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed760c-49bf-4e69-a4c4-cbc6e0346865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:52.110689 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed760c-49bf-4e69-a4c4-cbc6e0346865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:52.110962 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed760c-49bf-4e69-a4c4-cbc6e0346865 admin admin] PATH_INFO: `/v3/users/8ba4126b54a04efdb717aa174a65368c` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:52.112657 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed760c-49bf-4e69-a4c4-cbc6e0346865 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ba4126b54a04efdb717aa174a65368c)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:52.114625 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed760c-49bf-4e69-a4c4-cbc6e0346865 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:52.164553 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-39ed760c-49bf-4e69-a4c4-cbc6e0346865 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ba4126b54a04efdb717aa174a65368c'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:20:52.173053 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-39ed760c-49bf-4e69-a4c4-cbc6e0346865 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ba4126b54a04efdb717aa174a65368c'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:20:52.174732 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-39ed760c-49bf-4e69-a4c4-cbc6e0346865 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ba4126b54a04efdb717aa174a65368c'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:20:52.187118 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-39ed760c-49bf-4e69-a4c4-cbc6e0346865 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ba4126b54a04efdb717aa174a65368c'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:20:52.195807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-39ed760c-49bf-4e69-a4c4-cbc6e0346865 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ba4126b54a04efdb717aa174a65368c'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:20:52.201989 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1038/2109] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:20:52 2022] DELETE /identity/v3/users/8ba4126b54a04efdb717aa174a65368c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:20:52.456702 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f2fbd-ab1b-4ef3-bf3c-d45608cc5d0d service placement] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:52.468116 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f2fbd-ab1b-4ef3-bf3c-d45608cc5d0d service placement] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:52.547965 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c128e4c-6fb9-481c-9f8c-cab0a002537a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:52.549276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c128e4c-6fb9-481c-9f8c-cab0a002537a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:52.549561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c128e4c-6fb9-481c-9f8c-cab0a002537a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:52.549834 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c128e4c-6fb9-481c-9f8c-cab0a002537a admin admin] PATH_INFO: `/v3/projects/707b77c7fc104ae0853e057724a6d21e` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:52.551026 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c128e4c-6fb9-481c-9f8c-cab0a002537a admin admin] RBAC: Authorizing `identity:delete_project(project_id=707b77c7fc104ae0853e057724a6d21e)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:52.554291 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c128e4c-6fb9-481c-9f8c-cab0a002537a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:52.602997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2c128e4c-6fb9-481c-9f8c-cab0a002537a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '707b77c7fc104ae0853e057724a6d21e', 'request_id': 'req-2c128e4c-6fb9-481c-9f8c-cab0a002537a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:20:52.609110 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2c128e4c-6fb9-481c-9f8c-cab0a002537a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '707b77c7fc104ae0853e057724a6d21e', 'request_id': 'req-2c128e4c-6fb9-481c-9f8c-cab0a002537a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:20:52.616048 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2c128e4c-6fb9-481c-9f8c-cab0a002537a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '707b77c7fc104ae0853e057724a6d21e', 'request_id': 'req-2c128e4c-6fb9-481c-9f8c-cab0a002537a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:20:52.629855 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2c128e4c-6fb9-481c-9f8c-cab0a002537a admin admin] The token cache is being invalidate because project 707b77c7fc104ae0853e057724a6d21e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:20:52.630144 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2c128e4c-6fb9-481c-9f8c-cab0a002537a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:20:52.630792 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2c128e4c-6fb9-481c-9f8c-cab0a002537a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:20:52.632557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1072/2110] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:20:52 2022] DELETE /identity/v3/projects/707b77c7fc104ae0853e057724a6d21e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:20:52.636439 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed760c-49bf-4e69-a4c4-cbc6e0346865 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:52.638168 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-39ed760c-49bf-4e69-a4c4-cbc6e0346865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:52.651422 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed760c-49bf-4e69-a4c4-cbc6e0346865 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:52.656573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:20:52.721372 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23771a4c-d23a-4c75-af34-2ecdd8755559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:52.722107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23771a4c-d23a-4c75-af34-2ecdd8755559 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:52.722399 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23771a4c-d23a-4c75-af34-2ecdd8755559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:52.722824 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23771a4c-d23a-4c75-af34-2ecdd8755559 admin admin] PATH_INFO: `/v3/users/4511ccae6d494b9e861ba35eff80c093` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:52.724176 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23771a4c-d23a-4c75-af34-2ecdd8755559 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4511ccae6d494b9e861ba35eff80c093)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:52.726238 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23771a4c-d23a-4c75-af34-2ecdd8755559 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:52.785729 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-23771a4c-d23a-4c75-af34-2ecdd8755559 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4511ccae6d494b9e861ba35eff80c093'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:20:52.800074 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-23771a4c-d23a-4c75-af34-2ecdd8755559 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4511ccae6d494b9e861ba35eff80c093'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:20:52.800733 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-23771a4c-d23a-4c75-af34-2ecdd8755559 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4511ccae6d494b9e861ba35eff80c093'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:20:52.820002 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-23771a4c-d23a-4c75-af34-2ecdd8755559 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4511ccae6d494b9e861ba35eff80c093'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:20:52.831043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-23771a4c-d23a-4c75-af34-2ecdd8755559 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4511ccae6d494b9e861ba35eff80c093'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:20:52.860327 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1039/2111] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:20:52 2022] DELETE /identity/v3/users/4511ccae6d494b9e861ba35eff80c093 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:20:53.080586 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c128e4c-6fb9-481c-9f8c-cab0a002537a admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:53.082106 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2c128e4c-6fb9-481c-9f8c-cab0a002537a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:53.101436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c128e4c-6fb9-481c-9f8c-cab0a002537a admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:53.179488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:20:53.191933 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1b3cb-d45c-4cbc-b3fb-6daa535a7843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:53.193085 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1b3cb-d45c-4cbc-b3fb-6daa535a7843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:53.193862 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1b3cb-d45c-4cbc-b3fb-6daa535a7843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:53.194560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1b3cb-d45c-4cbc-b3fb-6daa535a7843 admin admin] PATH_INFO: `/v3/projects/e0f85e9a88ee42779f407c06304405bb` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:53.195899 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1b3cb-d45c-4cbc-b3fb-6daa535a7843 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0f85e9a88ee42779f407c06304405bb)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:53.198384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1b3cb-d45c-4cbc-b3fb-6daa535a7843 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:53.266061 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-eea1b3cb-d45c-4cbc-b3fb-6daa535a7843 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0f85e9a88ee42779f407c06304405bb', 'request_id': 'req-eea1b3cb-d45c-4cbc-b3fb-6daa535a7843'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:20:53.276521 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-eea1b3cb-d45c-4cbc-b3fb-6daa535a7843 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0f85e9a88ee42779f407c06304405bb', 'request_id': 'req-eea1b3cb-d45c-4cbc-b3fb-6daa535a7843'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:20:53.284614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-eea1b3cb-d45c-4cbc-b3fb-6daa535a7843 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0f85e9a88ee42779f407c06304405bb', 'request_id': 'req-eea1b3cb-d45c-4cbc-b3fb-6daa535a7843'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:20:53.302342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-eea1b3cb-d45c-4cbc-b3fb-6daa535a7843 admin admin] The token cache is being invalidate because project e0f85e9a88ee42779f407c06304405bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:20:53.303405 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-eea1b3cb-d45c-4cbc-b3fb-6daa535a7843 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:20:53.304056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-eea1b3cb-d45c-4cbc-b3fb-6daa535a7843 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:20:53.310550 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1073/2112] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:20:53 2022] DELETE /identity/v3/projects/e0f85e9a88ee42779f407c06304405bb => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:20:53.315883 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad6de5-c2d5-48ba-bab1-63d3e0bdc42b None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:53.316301 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad6de5-c2d5-48ba-bab1-63d3e0bdc42b None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:53.316465 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad6de5-c2d5-48ba-bab1-63d3e0bdc42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:53.377459 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-77ad6de5-c2d5-48ba-bab1-63d3e0bdc42b None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:20:53.379545 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-77ad6de5-c2d5-48ba-bab1-63d3e0bdc42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:53.432531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1b3cb-d45c-4cbc-b3fb-6daa535a7843 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:53.434133 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-eea1b3cb-d45c-4cbc-b3fb-6daa535a7843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:53.469959 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1040/2113] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:20:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:20:53.478906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc0b592-05bb-4aa8-8d96-25f5c8d27821 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:53.479260 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc0b592-05bb-4aa8-8d96-25f5c8d27821 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:53.479461 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc0b592-05bb-4aa8-8d96-25f5c8d27821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:53.503910 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-1bc0b592-05bb-4aa8-8d96-25f5c8d27821 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:20:53.504729 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-1bc0b592-05bb-4aa8-8d96-25f5c8d27821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:53.514883 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1041/2114] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:20:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:20:53.519585 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc0b592-05bb-4aa8-8d96-25f5c8d27821 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:53.522450 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1b3cb-d45c-4cbc-b3fb-6daa535a7843 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:53.529243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc0b592-05bb-4aa8-8d96-25f5c8d27821 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:53.534800 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37310522-729d-40ab-b9a4-6629ae0fd59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:53.535413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37310522-729d-40ab-b9a4-6629ae0fd59c admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:53.535720 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37310522-729d-40ab-b9a4-6629ae0fd59c admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:53.535848 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca19290-d52c-427e-a68a-cc3cc0ee469a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:53.536026 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37310522-729d-40ab-b9a4-6629ae0fd59c admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:53.536399 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca19290-d52c-427e-a68a-cc3cc0ee469a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:53.536654 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37310522-729d-40ab-b9a4-6629ae0fd59c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:20:53.536757 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca19290-d52c-427e-a68a-cc3cc0ee469a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:53.536934 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca19290-d52c-427e-a68a-cc3cc0ee469a admin admin] PATH_INFO: `/v3/users/6fb28ddd5511425a93c7b486d78a99eb` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:53.537039 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37310522-729d-40ab-b9a4-6629ae0fd59c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:53.538093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca19290-d52c-427e-a68a-cc3cc0ee469a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fb28ddd5511425a93c7b486d78a99eb)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:53.538580 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37310522-729d-40ab-b9a4-6629ae0fd59c admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:53.540045 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca19290-d52c-427e-a68a-cc3cc0ee469a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:53.551592 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1042/2115] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:20:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:20:53.555987 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37310522-729d-40ab-b9a4-6629ae0fd59c admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:53.570688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37310522-729d-40ab-b9a4-6629ae0fd59c admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:53.575772 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d289684-bc82-4992-9da7-03c9805182da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:53.576446 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d289684-bc82-4992-9da7-03c9805182da admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:53.576839 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d289684-bc82-4992-9da7-03c9805182da admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:53.577149 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d289684-bc82-4992-9da7-03c9805182da admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:53.577707 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d289684-bc82-4992-9da7-03c9805182da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:53.580430 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d289684-bc82-4992-9da7-03c9805182da admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:53.587810 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:20:53.595257 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-1ca19290-d52c-427e-a68a-cc3cc0ee469a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fb28ddd5511425a93c7b486d78a99eb'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:20:53.603676 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-1ca19290-d52c-427e-a68a-cc3cc0ee469a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fb28ddd5511425a93c7b486d78a99eb'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:20:53.605152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-1ca19290-d52c-427e-a68a-cc3cc0ee469a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fb28ddd5511425a93c7b486d78a99eb'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:20:53.606102 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-1ca19290-d52c-427e-a68a-cc3cc0ee469a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fb28ddd5511425a93c7b486d78a99eb'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:20:53.616055 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1043/2116] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:20:53 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:20:53.617396 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d289684-bc82-4992-9da7-03c9805182da admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:53.619339 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-6d289684-bc82-4992-9da7-03c9805182da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:53.619463 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-1ca19290-d52c-427e-a68a-cc3cc0ee469a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fb28ddd5511425a93c7b486d78a99eb'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:20:53.633837 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1074/2117] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:20:53 2022] DELETE /identity/v3/users/6fb28ddd5511425a93c7b486d78a99eb => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:20:53.689012 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d289684-bc82-4992-9da7-03c9805182da admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:53.753537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eb88d9-e85c-4003-8d05-327a7316d73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:53.754595 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb88d9-e85c-4003-8d05-327a7316d73d admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:53.754881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb88d9-e85c-4003-8d05-327a7316d73d admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:53.755150 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb88d9-e85c-4003-8d05-327a7316d73d admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:53.755850 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eb88d9-e85c-4003-8d05-327a7316d73d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:53.757798 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eb88d9-e85c-4003-8d05-327a7316d73d admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:53.786676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1044/2118] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:20:53 2022] POST /identity/v3/users => generated 293 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:20:53.795870 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca19290-d52c-427e-a68a-cc3cc0ee469a admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:53.807307 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca19290-d52c-427e-a68a-cc3cc0ee469a admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:53.813059 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f208d-b4b1-43b3-82ac-4376c04d931f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:53.813756 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f208d-b4b1-43b3-82ac-4376c04d931f admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:53.814042 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f208d-b4b1-43b3-82ac-4376c04d931f admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:53.814888 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f208d-b4b1-43b3-82ac-4376c04d931f admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:53.815722 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738f208d-b4b1-43b3-82ac-4376c04d931f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:53.817430 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738f208d-b4b1-43b3-82ac-4376c04d931f admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:53.819530 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:20:53.829097 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eb88d9-e85c-4003-8d05-327a7316d73d admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:53.831245 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-42eb88d9-e85c-4003-8d05-327a7316d73d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:53.834097 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1075/2119] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:20:53.837894 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f208d-b4b1-43b3-82ac-4376c04d931f admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:53.846714 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eb88d9-e85c-4003-8d05-327a7316d73d admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:53.851454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f208d-b4b1-43b3-82ac-4376c04d931f admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:53.851807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06af29-0b6c-484d-866e-00d0f036cffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:53.852439 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06af29-0b6c-484d-866e-00d0f036cffa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:53.852721 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06af29-0b6c-484d-866e-00d0f036cffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:53.853391 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06af29-0b6c-484d-866e-00d0f036cffa admin admin] PATH_INFO: `/v3/projects/09ba610aa04549c3ab792787257f4372` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:53.854575 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d06af29-0b6c-484d-866e-00d0f036cffa admin admin] RBAC: Authorizing `identity:delete_project(project_id=09ba610aa04549c3ab792787257f4372)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:53.856532 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d06af29-0b6c-484d-866e-00d0f036cffa admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:53.858224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f517841-8cef-4f17-a003-27b03d99b999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:53.859054 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f517841-8cef-4f17-a003-27b03d99b999 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:53.859338 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f517841-8cef-4f17-a003-27b03d99b999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:53.859791 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f517841-8cef-4f17-a003-27b03d99b999 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:53.860529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f517841-8cef-4f17-a003-27b03d99b999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:53.862746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f517841-8cef-4f17-a003-27b03d99b999 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:53.873355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1076/2120] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:20:53.878296 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f517841-8cef-4f17-a003-27b03d99b999 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:53.890460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f517841-8cef-4f17-a003-27b03d99b999 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:53.895342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c1598-c4fe-4763-8755-fc33119e85e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:53.896029 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c1598-c4fe-4763-8755-fc33119e85e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:53.896312 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c1598-c4fe-4763-8755-fc33119e85e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:53.896594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c1598-c4fe-4763-8755-fc33119e85e8 admin admin] PATH_INFO: `/v3/projects/32af7f5f27374094a857c76776b0014e/users/8a397a44ec734a38b5fc93071b0ac2e8/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:53.910631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3d06af29-0b6c-484d-866e-00d0f036cffa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09ba610aa04549c3ab792787257f4372', 'request_id': 'req-3d06af29-0b6c-484d-866e-00d0f036cffa'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:20:53.912307 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c1598-c4fe-4763-8755-fc33119e85e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32af7f5f27374094a857c76776b0014e, user_id=8a397a44ec734a38b5fc93071b0ac2e8, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:53.915134 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c1598-c4fe-4763-8755-fc33119e85e8 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:53.919181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3d06af29-0b6c-484d-866e-00d0f036cffa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09ba610aa04549c3ab792787257f4372', 'request_id': 'req-3d06af29-0b6c-484d-866e-00d0f036cffa'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:20:53.926092 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3d06af29-0b6c-484d-866e-00d0f036cffa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09ba610aa04549c3ab792787257f4372', 'request_id': 'req-3d06af29-0b6c-484d-866e-00d0f036cffa'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:20:53.930395 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1077/2121] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:20:53 2022] PUT /identity/v3/projects/32af7f5f27374094a857c76776b0014e/users/8a397a44ec734a38b5fc93071b0ac2e8/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:20:53.941324 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceaf647-4da5-4a3d-9990-acef1015876f None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:53.941607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceaf647-4da5-4a3d-9990-acef1015876f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:53.941887 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceaf647-4da5-4a3d-9990-acef1015876f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:53.945655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3d06af29-0b6c-484d-866e-00d0f036cffa admin admin] The token cache is being invalidate because project 09ba610aa04549c3ab792787257f4372 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:20:53.945943 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3d06af29-0b6c-484d-866e-00d0f036cffa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:20:53.946643 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3d06af29-0b6c-484d-866e-00d0f036cffa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:20:53.948383 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1045/2122] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:20:53 2022] DELETE /identity/v3/projects/09ba610aa04549c3ab792787257f4372 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:20:53.953783 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41b029-d29b-4344-bdc7-927e8e8269c6 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:53.954001 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41b029-d29b-4344-bdc7-927e8e8269c6 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:53.954276 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41b029-d29b-4344-bdc7-927e8e8269c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:53.968473 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:20:53.971092 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-aceaf647-4da5-4a3d-9990-acef1015876f None None] MFA Rules not processed for user `8a397a44ec734a38b5fc93071b0ac2e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:20:53.972352 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-aceaf647-4da5-4a3d-9990-acef1015876f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:53.976107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-6f41b029-d29b-4344-bdc7-927e8e8269c6 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:20:53.977750 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-6f41b029-d29b-4344-bdc7-927e8e8269c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:54.054299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1078/2123] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:20:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:20:54.056073 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1046/2124] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:20:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:20:54.063885 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a322d5-17bd-4b04-a840-dc8b9b682f1d None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:54.064202 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a322d5-17bd-4b04-a840-dc8b9b682f1d None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:54.064506 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a322d5-17bd-4b04-a840-dc8b9b682f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:54.067954 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f41b029-d29b-4344-bdc7-927e8e8269c6 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:54.069306 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-6f41b029-d29b-4344-bdc7-927e8e8269c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:54.084460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-17a322d5-17bd-4b04-a840-dc8b9b682f1d None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:20:54.085669 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-17a322d5-17bd-4b04-a840-dc8b9b682f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:54.094240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1079/2125] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:20:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:20:54.099013 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a322d5-17bd-4b04-a840-dc8b9b682f1d None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:54.110329 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a322d5-17bd-4b04-a840-dc8b9b682f1d None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:54.116863 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5cf9c3-6065-4270-b45b-6711b77af6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:54.117486 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5cf9c3-6065-4270-b45b-6711b77af6b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:54.117768 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5cf9c3-6065-4270-b45b-6711b77af6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:54.118382 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5cf9c3-6065-4270-b45b-6711b77af6b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:54.119350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5cf9c3-6065-4270-b45b-6711b77af6b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:20:54.119698 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5cf9c3-6065-4270-b45b-6711b77af6b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:54.121623 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5cf9c3-6065-4270-b45b-6711b77af6b8 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:54.134793 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f41b029-d29b-4344-bdc7-927e8e8269c6 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:54.135172 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1080/2126] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:20:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:20:54.138610 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e1be4-36bf-4073-831b-bf6eb3652050 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:54.139214 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e1be4-36bf-4073-831b-bf6eb3652050 service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:54.139356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5cf9c3-6065-4270-b45b-6711b77af6b8 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:54.139487 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e1be4-36bf-4073-831b-bf6eb3652050 service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:54.139781 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e1be4-36bf-4073-831b-bf6eb3652050 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:54.148260 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e1be4-36bf-4073-831b-bf6eb3652050 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:54.149042 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5cf9c3-6065-4270-b45b-6711b77af6b8 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:54.149938 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e1be4-36bf-4073-831b-bf6eb3652050 service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:54.155802 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a625c6-c2ef-4b43-8c76-717b0a71a1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:54.156418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a625c6-c2ef-4b43-8c76-717b0a71a1ba admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:54.156705 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a625c6-c2ef-4b43-8c76-717b0a71a1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:54.157003 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a625c6-c2ef-4b43-8c76-717b0a71a1ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:54.157635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a625c6-c2ef-4b43-8c76-717b0a71a1ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:54.160239 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a625c6-c2ef-4b43-8c76-717b0a71a1ba admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:54.160832 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1047/2127] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:20:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:20:54.194367 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1081/2128] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:20:54 2022] POST /identity/v3/projects => generated 391 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:20:54.196136 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e1be4-36bf-4073-831b-bf6eb3652050 service cinder] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:54.207149 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e1be4-36bf-4073-831b-bf6eb3652050 service cinder] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:54.272964 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609b198-13c7-4bff-b378-18c93addbd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:54.273575 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609b198-13c7-4bff-b378-18c93addbd16 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:54.273853 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609b198-13c7-4bff-b378-18c93addbd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:54.274119 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609b198-13c7-4bff-b378-18c93addbd16 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:54.275114 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609b198-13c7-4bff-b378-18c93addbd16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:54.277197 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609b198-13c7-4bff-b378-18c93addbd16 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:54.301416 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1048/2129] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:20:54 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:20:54.305285 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a625c6-c2ef-4b43-8c76-717b0a71a1ba admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:54.319295 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a625c6-c2ef-4b43-8c76-717b0a71a1ba admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:54.325290 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a89d3-2dc5-44f4-9083-a5d86876c7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:54.326469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a89d3-2dc5-44f4-9083-a5d86876c7ff admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:54.326873 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a89d3-2dc5-44f4-9083-a5d86876c7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:54.327212 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a89d3-2dc5-44f4-9083-a5d86876c7ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:54.327991 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a89d3-2dc5-44f4-9083-a5d86876c7ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:54.331180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a89d3-2dc5-44f4-9083-a5d86876c7ff admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:54.343376 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1082/2130] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:20:54.348555 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609b198-13c7-4bff-b378-18c93addbd16 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:54.359368 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609b198-13c7-4bff-b378-18c93addbd16 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:54.364501 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea0af8-7e56-4dec-9ec4-03dfb664326a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:54.365098 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea0af8-7e56-4dec-9ec4-03dfb664326a admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:54.365374 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea0af8-7e56-4dec-9ec4-03dfb664326a admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:54.365642 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea0af8-7e56-4dec-9ec4-03dfb664326a admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:54.366287 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea0af8-7e56-4dec-9ec4-03dfb664326a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:54.372113 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea0af8-7e56-4dec-9ec4-03dfb664326a admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:54.379405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1049/2131] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:20:54.384571 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a89d3-2dc5-44f4-9083-a5d86876c7ff admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:54.396626 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a89d3-2dc5-44f4-9083-a5d86876c7ff admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:54.404827 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df119090-4154-462a-a5f8-a781459b217b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:54.406121 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df119090-4154-462a-a5f8-a781459b217b admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:54.406415 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df119090-4154-462a-a5f8-a781459b217b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:54.406832 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df119090-4154-462a-a5f8-a781459b217b admin admin] PATH_INFO: `/v3/projects/938f765828c545a8840a6983bc5d9e6d/users/ff0999bd21bf4db5981053922200be2a/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:54.429276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df119090-4154-462a-a5f8-a781459b217b admin admin] RBAC: Authorizing `identity:create_grant(project_id=938f765828c545a8840a6983bc5d9e6d, user_id=ff0999bd21bf4db5981053922200be2a, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:54.432131 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df119090-4154-462a-a5f8-a781459b217b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:54.451461 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1083/2132] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:20:54 2022] PUT /identity/v3/projects/938f765828c545a8840a6983bc5d9e6d/users/ff0999bd21bf4db5981053922200be2a/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:20:54.469750 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24353e36-a403-4ee6-9f20-40f244b529a5 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:54.470165 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24353e36-a403-4ee6-9f20-40f244b529a5 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:54.471304 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24353e36-a403-4ee6-9f20-40f244b529a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:54.494121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-24353e36-a403-4ee6-9f20-40f244b529a5 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:20:54.495551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-24353e36-a403-4ee6-9f20-40f244b529a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:54.523719 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:20:54.565505 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1050/2133] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:20:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:20:54.575310 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df119090-4154-462a-a5f8-a781459b217b admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:54.577667 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-df119090-4154-462a-a5f8-a781459b217b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:54.612668 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:20:54.635678 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df119090-4154-462a-a5f8-a781459b217b admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:54.639462 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a9fe16-e4d2-47ad-9869-a7c99c0a6c14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:54.640097 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9fe16-e4d2-47ad-9869-a7c99c0a6c14 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:54.640372 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9fe16-e4d2-47ad-9869-a7c99c0a6c14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:54.640706 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9fe16-e4d2-47ad-9869-a7c99c0a6c14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:54.652348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a9fe16-e4d2-47ad-9869-a7c99c0a6c14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:54.653681 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a9fe16-e4d2-47ad-9869-a7c99c0a6c14 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:54.665102 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1084/2134] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:20:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:20:58.966290 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665c4aa-553c-4f7b-8669-452a0f069145 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:58.966695 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665c4aa-553c-4f7b-8669-452a0f069145 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:58.966891 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665c4aa-553c-4f7b-8669-452a0f069145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:59.004826 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-8665c4aa-553c-4f7b-8669-452a0f069145 None None] MFA Rules not processed for user `ff0999bd21bf4db5981053922200be2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:20:59.006446 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-8665c4aa-553c-4f7b-8669-452a0f069145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:59.074528 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1051/2135] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:20:58 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:20:59.085392 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366044fc-6b81-4603-80a7-b5b235de1498 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:59.085392 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366044fc-6b81-4603-80a7-b5b235de1498 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:59.085392 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366044fc-6b81-4603-80a7-b5b235de1498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:59.108937 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-366044fc-6b81-4603-80a7-b5b235de1498 None None] MFA Rules not processed for user `ff0999bd21bf4db5981053922200be2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:20:59.110861 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-366044fc-6b81-4603-80a7-b5b235de1498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:59.127651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1085/2136] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:20:59 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:20:59.145472 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665c4aa-553c-4f7b-8669-452a0f069145 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:59.146949 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-8665c4aa-553c-4f7b-8669-452a0f069145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:59.225104 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665c4aa-553c-4f7b-8669-452a0f069145 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:59.230596 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fef8cb-94e2-45c4-9d43-249b97a960a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:59.231237 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fef8cb-94e2-45c4-9d43-249b97a960a2 service nova] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:59.231508 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fef8cb-94e2-45c4-9d43-249b97a960a2 service nova] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:59.231817 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fef8cb-94e2-45c4-9d43-249b97a960a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:59.240810 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fef8cb-94e2-45c4-9d43-249b97a960a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:59.242407 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fef8cb-94e2-45c4-9d43-249b97a960a2 service nova] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:59.251737 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1052/2137] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:20:59 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:20:59.966729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258f2b4-06bc-4fd7-9509-1c5becda729b None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:59.967193 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258f2b4-06bc-4fd7-9509-1c5becda729b None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:59.967676 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258f2b4-06bc-4fd7-9509-1c5becda729b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:59.989666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-0258f2b4-06bc-4fd7-9509-1c5becda729b None None] MFA Rules not processed for user `ff0999bd21bf4db5981053922200be2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:20:59.990967 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-0258f2b4-06bc-4fd7-9509-1c5becda729b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:59.999004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1086/2138] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:20:59 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:09.247835 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=80 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:09.983840 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=17 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:12.855095 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fef8cb-94e2-45c4-9d43-249b97a960a2 service nova] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:12.856590 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-55fef8cb-94e2-45c4-9d43-249b97a960a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:12.870755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fef8cb-94e2-45c4-9d43-249b97a960a2 service nova] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:12.875334 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02361c43-c577-442c-b2f0-ead9fe0533ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:12.875998 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02361c43-c577-442c-b2f0-ead9fe0533ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:12.876283 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02361c43-c577-442c-b2f0-ead9fe0533ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:12.876819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02361c43-c577-442c-b2f0-ead9fe0533ca admin admin] PATH_INFO: `/v3/users/8a397a44ec734a38b5fc93071b0ac2e8` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:12.877728 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02361c43-c577-442c-b2f0-ead9fe0533ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a397a44ec734a38b5fc93071b0ac2e8)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:12.879701 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02361c43-c577-442c-b2f0-ead9fe0533ca admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:12.932256 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-02361c43-c577-442c-b2f0-ead9fe0533ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a397a44ec734a38b5fc93071b0ac2e8'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:12.939591 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-02361c43-c577-442c-b2f0-ead9fe0533ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a397a44ec734a38b5fc93071b0ac2e8'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:12.940242 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-02361c43-c577-442c-b2f0-ead9fe0533ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a397a44ec734a38b5fc93071b0ac2e8'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:12.952494 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-02361c43-c577-442c-b2f0-ead9fe0533ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a397a44ec734a38b5fc93071b0ac2e8'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:12.959914 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-02361c43-c577-442c-b2f0-ead9fe0533ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a397a44ec734a38b5fc93071b0ac2e8'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:12.968518 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1053/2139] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:21:12 2022] DELETE /identity/v3/users/8a397a44ec734a38b5fc93071b0ac2e8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:12.973565 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34983e0c-1a0f-4d50-9bb0-24668ba02238 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:12.974083 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34983e0c-1a0f-4d50-9bb0-24668ba02238 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:12.974083 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34983e0c-1a0f-4d50-9bb0-24668ba02238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:12.993245 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-34983e0c-1a0f-4d50-9bb0-24668ba02238 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:12.994442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-34983e0c-1a0f-4d50-9bb0-24668ba02238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:13.053314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1087/2140] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:13.063222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02361c43-c577-442c-b2f0-ead9fe0533ca admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:13.064586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-02361c43-c577-442c-b2f0-ead9fe0533ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:13.123126 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02361c43-c577-442c-b2f0-ead9fe0533ca admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:13.127351 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac87919-54ff-41a7-82fd-f4fec82aec52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:13.127990 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac87919-54ff-41a7-82fd-f4fec82aec52 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:13.128264 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac87919-54ff-41a7-82fd-f4fec82aec52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:13.128539 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac87919-54ff-41a7-82fd-f4fec82aec52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:13.138087 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac87919-54ff-41a7-82fd-f4fec82aec52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:13.139552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac87919-54ff-41a7-82fd-f4fec82aec52 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:13.150452 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1054/2141] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:21:13.424729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34983e0c-1a0f-4d50-9bb0-24668ba02238 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:13.432729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-34983e0c-1a0f-4d50-9bb0-24668ba02238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:13.445548 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34983e0c-1a0f-4d50-9bb0-24668ba02238 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:13.450346 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0022fd-7539-4a49-9927-46d6b5be657a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:13.450986 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0022fd-7539-4a49-9927-46d6b5be657a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:13.451401 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0022fd-7539-4a49-9927-46d6b5be657a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:13.451710 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0022fd-7539-4a49-9927-46d6b5be657a admin admin] PATH_INFO: `/v3/projects/32af7f5f27374094a857c76776b0014e` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:13.452849 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0022fd-7539-4a49-9927-46d6b5be657a admin admin] RBAC: Authorizing `identity:delete_project(project_id=32af7f5f27374094a857c76776b0014e)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:13.454749 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0022fd-7539-4a49-9927-46d6b5be657a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:13.505889 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0c0022fd-7539-4a49-9927-46d6b5be657a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32af7f5f27374094a857c76776b0014e', 'request_id': 'req-0c0022fd-7539-4a49-9927-46d6b5be657a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:13.511318 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0c0022fd-7539-4a49-9927-46d6b5be657a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32af7f5f27374094a857c76776b0014e', 'request_id': 'req-0c0022fd-7539-4a49-9927-46d6b5be657a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:13.517472 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0c0022fd-7539-4a49-9927-46d6b5be657a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32af7f5f27374094a857c76776b0014e', 'request_id': 'req-0c0022fd-7539-4a49-9927-46d6b5be657a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:13.530270 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0c0022fd-7539-4a49-9927-46d6b5be657a admin admin] The token cache is being invalidate because project 32af7f5f27374094a857c76776b0014e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:21:13.530560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0c0022fd-7539-4a49-9927-46d6b5be657a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:13.531211 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0c0022fd-7539-4a49-9927-46d6b5be657a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:13.532711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1088/2142] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:21:13 2022] DELETE /identity/v3/projects/32af7f5f27374094a857c76776b0014e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:13.537978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c838b-997b-40db-a83c-f40a8d43ca65 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:13.538249 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c838b-997b-40db-a83c-f40a8d43ca65 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:13.538521 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c838b-997b-40db-a83c-f40a8d43ca65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:13.559219 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-a87c838b-997b-40db-a83c-f40a8d43ca65 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:13.560524 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-a87c838b-997b-40db-a83c-f40a8d43ca65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:13.620113 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1055/2143] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:13.629000 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f56d4-71dd-4e08-beab-3cb8b488e600 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:13.629290 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f56d4-71dd-4e08-beab-3cb8b488e600 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:13.629567 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f56d4-71dd-4e08-beab-3cb8b488e600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:13.650528 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-ea7f56d4-71dd-4e08-beab-3cb8b488e600 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:13.651789 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ea7f56d4-71dd-4e08-beab-3cb8b488e600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:13.659837 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1089/2144] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:13.664170 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87c838b-997b-40db-a83c-f40a8d43ca65 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:13.674118 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87c838b-997b-40db-a83c-f40a8d43ca65 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:13.678727 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32139752-9a0a-4edc-a725-849fb4e634fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:13.679335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32139752-9a0a-4edc-a725-849fb4e634fd admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:13.679644 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32139752-9a0a-4edc-a725-849fb4e634fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:13.679920 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32139752-9a0a-4edc-a725-849fb4e634fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:13.680573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32139752-9a0a-4edc-a725-849fb4e634fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:21:13.680876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32139752-9a0a-4edc-a725-849fb4e634fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:13.682471 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32139752-9a0a-4edc-a725-849fb4e634fd admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:13.695412 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1056/2145] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:21:13.700118 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f56d4-71dd-4e08-beab-3cb8b488e600 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:13.710248 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f56d4-71dd-4e08-beab-3cb8b488e600 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:13.715146 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8ddca-1c88-4d21-8f6d-5d1bccae8732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:13.715789 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8ddca-1c88-4d21-8f6d-5d1bccae8732 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:13.716138 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8ddca-1c88-4d21-8f6d-5d1bccae8732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:13.716336 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8ddca-1c88-4d21-8f6d-5d1bccae8732 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:13.716981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b8ddca-1c88-4d21-8f6d-5d1bccae8732 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:13.718900 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b8ddca-1c88-4d21-8f6d-5d1bccae8732 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:13.747275 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1090/2146] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:21:13 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:21:13.751478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32139752-9a0a-4edc-a725-849fb4e634fd admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:13.759195 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:13.766079 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32139752-9a0a-4edc-a725-849fb4e634fd admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:13.825766 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f3b82-987b-4a62-8be5-6f3301be6b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:13.826361 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f3b82-987b-4a62-8be5-6f3301be6b39 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:13.826637 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f3b82-987b-4a62-8be5-6f3301be6b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:13.826907 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f3b82-987b-4a62-8be5-6f3301be6b39 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:13.827561 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f3b82-987b-4a62-8be5-6f3301be6b39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:13.829469 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f3b82-987b-4a62-8be5-6f3301be6b39 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:13.850723 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1057/2147] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:21:13 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:21:13.854809 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8ddca-1c88-4d21-8f6d-5d1bccae8732 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:13.865495 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8ddca-1c88-4d21-8f6d-5d1bccae8732 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:13.870111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846bedbf-587c-47fd-9317-18e36cf563a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:13.870761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bedbf-587c-47fd-9317-18e36cf563a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:13.870987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bedbf-587c-47fd-9317-18e36cf563a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:13.871262 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bedbf-587c-47fd-9317-18e36cf563a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:13.871941 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846bedbf-587c-47fd-9317-18e36cf563a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:13.873532 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846bedbf-587c-47fd-9317-18e36cf563a4 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:13.883951 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1091/2148] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:21:13.887869 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f3b82-987b-4a62-8be5-6f3301be6b39 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:13.897789 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f3b82-987b-4a62-8be5-6f3301be6b39 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:13.902807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56b587-53d0-432e-ae7d-fc36a1a1cbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:13.903383 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56b587-53d0-432e-ae7d-fc36a1a1cbbb admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:13.903694 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56b587-53d0-432e-ae7d-fc36a1a1cbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:13.903967 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56b587-53d0-432e-ae7d-fc36a1a1cbbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:13.904631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c56b587-53d0-432e-ae7d-fc36a1a1cbbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:13.906254 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c56b587-53d0-432e-ae7d-fc36a1a1cbbb admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:13.917014 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1058/2149] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:21:13.921104 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846bedbf-587c-47fd-9317-18e36cf563a4 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:13.931418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846bedbf-587c-47fd-9317-18e36cf563a4 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:13.936029 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766c161-b872-4e3b-898a-71e0be446e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:13.936668 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766c161-b872-4e3b-898a-71e0be446e62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:13.936951 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766c161-b872-4e3b-898a-71e0be446e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:13.937227 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766c161-b872-4e3b-898a-71e0be446e62 admin admin] PATH_INFO: `/v3/projects/8689e09a1efd49a6bad89984909b185a/users/64b1a765aaf04a649dd37d748c2c57f1/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:13.952088 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766c161-b872-4e3b-898a-71e0be446e62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8689e09a1efd49a6bad89984909b185a, user_id=64b1a765aaf04a649dd37d748c2c57f1, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:13.954751 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766c161-b872-4e3b-898a-71e0be446e62 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:13.968547 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1092/2150] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:21:13 2022] PUT /identity/v3/projects/8689e09a1efd49a6bad89984909b185a/users/64b1a765aaf04a649dd37d748c2c57f1/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:13.979213 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e71a0-c860-4118-9efd-8902e3a67ef9 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:13.979629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e71a0-c860-4118-9efd-8902e3a67ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:13.979979 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e71a0-c860-4118-9efd-8902e3a67ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:14.011495 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-f11e71a0-c860-4118-9efd-8902e3a67ef9 None None] MFA Rules not processed for user `64b1a765aaf04a649dd37d748c2c57f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:14.012949 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-f11e71a0-c860-4118-9efd-8902e3a67ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:14.082269 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1059/2151] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:14.093497 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766c161-b872-4e3b-898a-71e0be446e62 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:14.095270 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3766c161-b872-4e3b-898a-71e0be446e62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:14.118621 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:14.167953 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766c161-b872-4e3b-898a-71e0be446e62 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:14.171490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9cf95c-4380-4537-9390-2cd8f35733c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:14.172122 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cf95c-4380-4537-9390-2cd8f35733c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:14.172399 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cf95c-4380-4537-9390-2cd8f35733c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:14.172664 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cf95c-4380-4537-9390-2cd8f35733c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:14.181172 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9cf95c-4380-4537-9390-2cd8f35733c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:14.182606 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9cf95c-4380-4537-9390-2cd8f35733c0 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:14.192109 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1093/2152] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:21:21.927833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e71a0-c860-4118-9efd-8902e3a67ef9 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:21.929856 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-f11e71a0-c860-4118-9efd-8902e3a67ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:21.996575 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e71a0-c860-4118-9efd-8902e3a67ef9 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:22.001104 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70cad8-494e-421c-a4ee-d42005d21e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:22.001728 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70cad8-494e-421c-a4ee-d42005d21e4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:22.002036 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70cad8-494e-421c-a4ee-d42005d21e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:22.002313 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70cad8-494e-421c-a4ee-d42005d21e4e admin admin] PATH_INFO: `/v3/users/07b88b4429d74b16921f224a3427d669` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:22.003466 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70cad8-494e-421c-a4ee-d42005d21e4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=07b88b4429d74b16921f224a3427d669)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:22.005415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70cad8-494e-421c-a4ee-d42005d21e4e admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:22.017632 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:22.051748 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-bc70cad8-494e-421c-a4ee-d42005d21e4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07b88b4429d74b16921f224a3427d669'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:22.059633 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-bc70cad8-494e-421c-a4ee-d42005d21e4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07b88b4429d74b16921f224a3427d669'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:22.060272 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-bc70cad8-494e-421c-a4ee-d42005d21e4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07b88b4429d74b16921f224a3427d669'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:22.072146 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-bc70cad8-494e-421c-a4ee-d42005d21e4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07b88b4429d74b16921f224a3427d669'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:22.078975 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-bc70cad8-494e-421c-a4ee-d42005d21e4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07b88b4429d74b16921f224a3427d669'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:22.085423 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1060/2153] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:21:21 2022] DELETE /identity/v3/users/07b88b4429d74b16921f224a3427d669 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:21:22.089933 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f013b1b-5f06-45d1-a0a5-5aa52b1881c9 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:22.090417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f013b1b-5f06-45d1-a0a5-5aa52b1881c9 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:22.090512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f013b1b-5f06-45d1-a0a5-5aa52b1881c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:22.110359 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-7f013b1b-5f06-45d1-a0a5-5aa52b1881c9 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:22.111642 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7f013b1b-5f06-45d1-a0a5-5aa52b1881c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:22.175546 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1094/2154] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:22.186210 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70cad8-494e-421c-a4ee-d42005d21e4e admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:22.187649 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-bc70cad8-494e-421c-a4ee-d42005d21e4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:22.247896 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70cad8-494e-421c-a4ee-d42005d21e4e admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:22.251904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f508c6f-01d6-4ab9-a63f-e499da862d94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:22.252505 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f508c6f-01d6-4ab9-a63f-e499da862d94 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:22.252785 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f508c6f-01d6-4ab9-a63f-e499da862d94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:22.253550 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f508c6f-01d6-4ab9-a63f-e499da862d94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:22.263356 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f508c6f-01d6-4ab9-a63f-e499da862d94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:22.264834 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f508c6f-01d6-4ab9-a63f-e499da862d94 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:22.275270 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1061/2155] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:21:22.599739 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f013b1b-5f06-45d1-a0a5-5aa52b1881c9 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:22.601271 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7f013b1b-5f06-45d1-a0a5-5aa52b1881c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:22.621081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f013b1b-5f06-45d1-a0a5-5aa52b1881c9 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:22.625781 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9765f-6ea3-4dbc-a8bd-d9b284fc7f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:22.626419 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9765f-6ea3-4dbc-a8bd-d9b284fc7f51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:22.626698 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9765f-6ea3-4dbc-a8bd-d9b284fc7f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:22.627489 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9765f-6ea3-4dbc-a8bd-d9b284fc7f51 admin admin] PATH_INFO: `/v3/projects/7587d39b1b0143c39b17d29995b4a8dc` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:22.628668 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d9765f-6ea3-4dbc-a8bd-d9b284fc7f51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7587d39b1b0143c39b17d29995b4a8dc)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:22.630630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d9765f-6ea3-4dbc-a8bd-d9b284fc7f51 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:22.680921 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e1d9765f-6ea3-4dbc-a8bd-d9b284fc7f51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7587d39b1b0143c39b17d29995b4a8dc', 'request_id': 'req-e1d9765f-6ea3-4dbc-a8bd-d9b284fc7f51'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:22.686376 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e1d9765f-6ea3-4dbc-a8bd-d9b284fc7f51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7587d39b1b0143c39b17d29995b4a8dc', 'request_id': 'req-e1d9765f-6ea3-4dbc-a8bd-d9b284fc7f51'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:22.692989 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e1d9765f-6ea3-4dbc-a8bd-d9b284fc7f51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7587d39b1b0143c39b17d29995b4a8dc', 'request_id': 'req-e1d9765f-6ea3-4dbc-a8bd-d9b284fc7f51'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:22.705988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e1d9765f-6ea3-4dbc-a8bd-d9b284fc7f51 admin admin] The token cache is being invalidate because project 7587d39b1b0143c39b17d29995b4a8dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:21:22.706272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e1d9765f-6ea3-4dbc-a8bd-d9b284fc7f51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:22.706917 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e1d9765f-6ea3-4dbc-a8bd-d9b284fc7f51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:22.708420 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1095/2156] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:21:22 2022] DELETE /identity/v3/projects/7587d39b1b0143c39b17d29995b4a8dc => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:22.713395 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53441f31-159a-4dad-8f8b-0a3ee1eb63cc None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:22.713627 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53441f31-159a-4dad-8f8b-0a3ee1eb63cc None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:22.713917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53441f31-159a-4dad-8f8b-0a3ee1eb63cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:22.733492 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-53441f31-159a-4dad-8f8b-0a3ee1eb63cc None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:22.734698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-53441f31-159a-4dad-8f8b-0a3ee1eb63cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:22.801969 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1062/2157] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:21:22.810996 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79833e-fdcc-4f07-8329-4a15210e9933 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:22.811277 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79833e-fdcc-4f07-8329-4a15210e9933 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:22.811578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79833e-fdcc-4f07-8329-4a15210e9933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:22.836420 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-3b79833e-fdcc-4f07-8329-4a15210e9933 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:22.837689 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3b79833e-fdcc-4f07-8329-4a15210e9933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:22.849027 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1096/2158] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:22.853332 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53441f31-159a-4dad-8f8b-0a3ee1eb63cc None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:22.863819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53441f31-159a-4dad-8f8b-0a3ee1eb63cc None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:22.868932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db601bb4-7d45-4da6-93b0-f351e85d4522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:22.879446 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db601bb4-7d45-4da6-93b0-f351e85d4522 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:22.879908 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db601bb4-7d45-4da6-93b0-f351e85d4522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:22.880321 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db601bb4-7d45-4da6-93b0-f351e85d4522 admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:22.885809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db601bb4-7d45-4da6-93b0-f351e85d4522 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:21:22.886121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db601bb4-7d45-4da6-93b0-f351e85d4522 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:22.887816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db601bb4-7d45-4da6-93b0-f351e85d4522 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:22.900626 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1063/2159] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:21:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:21:22.905122 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79833e-fdcc-4f07-8329-4a15210e9933 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:22.915060 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79833e-fdcc-4f07-8329-4a15210e9933 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:22.919856 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c45625-ddce-4e0a-8676-a909aa386ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:22.920448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c45625-ddce-4e0a-8676-a909aa386ef7 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:22.920727 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c45625-ddce-4e0a-8676-a909aa386ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:22.920998 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c45625-ddce-4e0a-8676-a909aa386ef7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:22.921622 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c45625-ddce-4e0a-8676-a909aa386ef7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:22.923515 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c45625-ddce-4e0a-8676-a909aa386ef7 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:22.950862 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1097/2160] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:21:22 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:21:22.955027 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db601bb4-7d45-4da6-93b0-f351e85d4522 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:22.965080 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db601bb4-7d45-4da6-93b0-f351e85d4522 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:22.981916 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:23.026238 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8cd446-2157-43f0-8af5-502949c13e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:23.027124 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8cd446-2157-43f0-8af5-502949c13e95 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:23.027124 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8cd446-2157-43f0-8af5-502949c13e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:23.027365 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8cd446-2157-43f0-8af5-502949c13e95 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:23.028039 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8cd446-2157-43f0-8af5-502949c13e95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:23.029963 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8cd446-2157-43f0-8af5-502949c13e95 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:23.055581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1064/2161] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:21:22 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:21:23.059701 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c45625-ddce-4e0a-8676-a909aa386ef7 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:23.072884 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c45625-ddce-4e0a-8676-a909aa386ef7 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:23.077619 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a04c1-79fb-4ed9-86e7-50b87c0ff4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:23.078215 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a04c1-79fb-4ed9-86e7-50b87c0ff4d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:23.078497 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a04c1-79fb-4ed9-86e7-50b87c0ff4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:23.078776 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a04c1-79fb-4ed9-86e7-50b87c0ff4d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:23.079416 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a04c1-79fb-4ed9-86e7-50b87c0ff4d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:23.081238 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a04c1-79fb-4ed9-86e7-50b87c0ff4d9 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:23.091714 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1098/2162] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:21:23.096532 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8cd446-2157-43f0-8af5-502949c13e95 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:23.107018 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8cd446-2157-43f0-8af5-502949c13e95 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:23.112240 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba3d3c-dbd3-4839-a5ad-3ef68416fdf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:23.112839 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba3d3c-dbd3-4839-a5ad-3ef68416fdf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:23.113111 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba3d3c-dbd3-4839-a5ad-3ef68416fdf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:23.113388 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba3d3c-dbd3-4839-a5ad-3ef68416fdf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:23.114026 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ba3d3c-dbd3-4839-a5ad-3ef68416fdf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:23.115751 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ba3d3c-dbd3-4839-a5ad-3ef68416fdf2 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:23.128579 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1065/2163] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:21:23.132923 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a04c1-79fb-4ed9-86e7-50b87c0ff4d9 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:23.143330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a04c1-79fb-4ed9-86e7-50b87c0ff4d9 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:23.148012 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4f398-fcaa-468a-b29c-af24af5c5074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:23.148660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4f398-fcaa-468a-b29c-af24af5c5074 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:23.148937 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4f398-fcaa-468a-b29c-af24af5c5074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:23.149208 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4f398-fcaa-468a-b29c-af24af5c5074 admin admin] PATH_INFO: `/v3/projects/629a461f93fc4e77ba73c47b66cc7814/users/7e5b55cdebe34ea7ac7d26986e1e4ea2/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:23.151713 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:23.168283 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b4f398-fcaa-468a-b29c-af24af5c5074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=629a461f93fc4e77ba73c47b66cc7814, user_id=7e5b55cdebe34ea7ac7d26986e1e4ea2, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:23.171055 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b4f398-fcaa-468a-b29c-af24af5c5074 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:23.191899 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1099/2164] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:21:23 2022] PUT /identity/v3/projects/629a461f93fc4e77ba73c47b66cc7814/users/7e5b55cdebe34ea7ac7d26986e1e4ea2/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:23.202174 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca19708-6532-4c53-9a69-5854ba8b100b None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:23.202513 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca19708-6532-4c53-9a69-5854ba8b100b None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:23.202736 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca19708-6532-4c53-9a69-5854ba8b100b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:23.243067 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-bca19708-6532-4c53-9a69-5854ba8b100b None None] MFA Rules not processed for user `7e5b55cdebe34ea7ac7d26986e1e4ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:23.244658 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-bca19708-6532-4c53-9a69-5854ba8b100b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:23.325354 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1066/2165] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:23.331039 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4f398-fcaa-468a-b29c-af24af5c5074 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:23.409082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4f398-fcaa-468a-b29c-af24af5c5074 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:23.414017 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332da0bb-1ecd-4292-b5f1-5a6597032fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:23.414767 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332da0bb-1ecd-4292-b5f1-5a6597032fbb admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:23.415182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332da0bb-1ecd-4292-b5f1-5a6597032fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:23.415384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332da0bb-1ecd-4292-b5f1-5a6597032fbb admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:23.416169 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332da0bb-1ecd-4292-b5f1-5a6597032fbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:23.418011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332da0bb-1ecd-4292-b5f1-5a6597032fbb admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:23.448977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1100/2166] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:21:23 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:21:23.452915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca19708-6532-4c53-9a69-5854ba8b100b None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:23.464630 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca19708-6532-4c53-9a69-5854ba8b100b None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:23.524213 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb4da3-9c94-4505-a3db-72aeb37b030a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:23.525078 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb4da3-9c94-4505-a3db-72aeb37b030a admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:23.525356 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb4da3-9c94-4505-a3db-72aeb37b030a admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:23.525627 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb4da3-9c94-4505-a3db-72aeb37b030a admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:23.526789 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb4da3-9c94-4505-a3db-72aeb37b030a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:23.528804 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb4da3-9c94-4505-a3db-72aeb37b030a admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:23.555619 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1067/2167] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:21:23 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:21:23.558206 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332da0bb-1ecd-4292-b5f1-5a6597032fbb admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:23.569142 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332da0bb-1ecd-4292-b5f1-5a6597032fbb admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:23.574047 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0f75c-522c-420f-9f36-13d431d93e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:23.574747 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0f75c-522c-420f-9f36-13d431d93e2e admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:23.575108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0f75c-522c-420f-9f36-13d431d93e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:23.575445 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0f75c-522c-420f-9f36-13d431d93e2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:23.576128 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d0f75c-522c-420f-9f36-13d431d93e2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:23.577726 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d0f75c-522c-420f-9f36-13d431d93e2e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:23.591583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1101/2168] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:21:23.593035 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb4da3-9c94-4505-a3db-72aeb37b030a admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:23.603750 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb4da3-9c94-4505-a3db-72aeb37b030a admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:23.608931 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1349fcd0-74e2-4e16-9554-35063c83a7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:23.609544 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349fcd0-74e2-4e16-9554-35063c83a7a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:23.609810 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349fcd0-74e2-4e16-9554-35063c83a7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:23.610083 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349fcd0-74e2-4e16-9554-35063c83a7a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:23.610724 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1349fcd0-74e2-4e16-9554-35063c83a7a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:23.612393 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1349fcd0-74e2-4e16-9554-35063c83a7a5 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:23.622287 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1068/2169] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:21:23.626467 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0f75c-522c-420f-9f36-13d431d93e2e admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:23.636282 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0f75c-522c-420f-9f36-13d431d93e2e admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:23.640852 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e75f57-a4c4-4c1a-b057-590e41ec0f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:23.641490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e75f57-a4c4-4c1a-b057-590e41ec0f73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:23.641763 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e75f57-a4c4-4c1a-b057-590e41ec0f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:23.642031 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e75f57-a4c4-4c1a-b057-590e41ec0f73 admin admin] PATH_INFO: `/v3/projects/8a64774d68864e4fbf1e19f32be643bb/users/241425bea3504f98af1f8a6d23617b7f/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:23.656970 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e75f57-a4c4-4c1a-b057-590e41ec0f73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a64774d68864e4fbf1e19f32be643bb, user_id=241425bea3504f98af1f8a6d23617b7f, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:23.660316 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e75f57-a4c4-4c1a-b057-590e41ec0f73 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:23.673942 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1102/2170] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:21:23 2022] PUT /identity/v3/projects/8a64774d68864e4fbf1e19f32be643bb/users/241425bea3504f98af1f8a6d23617b7f/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:23.684151 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda19db-10fb-46a5-b597-06d90a9754f9 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:23.684419 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda19db-10fb-46a5-b597-06d90a9754f9 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:23.684696 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda19db-10fb-46a5-b597-06d90a9754f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:23.698721 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:23.714282 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-8eda19db-10fb-46a5-b597-06d90a9754f9 None None] MFA Rules not processed for user `241425bea3504f98af1f8a6d23617b7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:23.715566 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-8eda19db-10fb-46a5-b597-06d90a9754f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:23.781089 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1069/2171] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:23.786406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e75f57-a4c4-4c1a-b057-590e41ec0f73 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:23.849550 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e75f57-a4c4-4c1a-b057-590e41ec0f73 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:23.854198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b923191-0441-4c16-8e94-afee62f4f1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:23.854832 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b923191-0441-4c16-8e94-afee62f4f1ac admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:23.855087 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b923191-0441-4c16-8e94-afee62f4f1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:23.855365 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b923191-0441-4c16-8e94-afee62f4f1ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:23.856044 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b923191-0441-4c16-8e94-afee62f4f1ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:23.857952 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b923191-0441-4c16-8e94-afee62f4f1ac admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:23.884295 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1103/2172] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:21:23 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:21:23.888356 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda19db-10fb-46a5-b597-06d90a9754f9 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:23.901784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda19db-10fb-46a5-b597-06d90a9754f9 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:23.958245 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f1605-6bb8-4442-aa61-0dba1fbb0dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:23.958890 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f1605-6bb8-4442-aa61-0dba1fbb0dba admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:23.959115 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f1605-6bb8-4442-aa61-0dba1fbb0dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:23.959390 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f1605-6bb8-4442-aa61-0dba1fbb0dba admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:23.960063 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f1605-6bb8-4442-aa61-0dba1fbb0dba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:23.961985 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f1605-6bb8-4442-aa61-0dba1fbb0dba admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:23.986337 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1070/2173] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:21:23 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:21:23.989282 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b923191-0441-4c16-8e94-afee62f4f1ac admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:23.999439 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b923191-0441-4c16-8e94-afee62f4f1ac admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:24.004211 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3bdc0c-9c7c-4b90-804f-36870230590e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:24.004829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bdc0c-9c7c-4b90-804f-36870230590e admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:24.005111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bdc0c-9c7c-4b90-804f-36870230590e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:24.005502 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bdc0c-9c7c-4b90-804f-36870230590e admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:24.006076 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3bdc0c-9c7c-4b90-804f-36870230590e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:24.007871 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3bdc0c-9c7c-4b90-804f-36870230590e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:24.024795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1104/2174] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:21:24.028991 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f1605-6bb8-4442-aa61-0dba1fbb0dba admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:24.040092 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f1605-6bb8-4442-aa61-0dba1fbb0dba admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:24.045563 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f812b1e-f1cc-43ca-a9c9-aaf12e848e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:24.046217 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f812b1e-f1cc-43ca-a9c9-aaf12e848e71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:24.046519 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f812b1e-f1cc-43ca-a9c9-aaf12e848e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:24.046791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f812b1e-f1cc-43ca-a9c9-aaf12e848e71 admin admin] PATH_INFO: `/v3/projects/606b96a009524ac0908d37fce333a84f/users/217bce795ae04ee280071224db6d26d4/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:24.063356 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f812b1e-f1cc-43ca-a9c9-aaf12e848e71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=606b96a009524ac0908d37fce333a84f, user_id=217bce795ae04ee280071224db6d26d4, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:24.066127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f812b1e-f1cc-43ca-a9c9-aaf12e848e71 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:24.079812 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1071/2175] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:21:24 2022] PUT /identity/v3/projects/606b96a009524ac0908d37fce333a84f/users/217bce795ae04ee280071224db6d26d4/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:24.083843 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3bdc0c-9c7c-4b90-804f-36870230590e admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:24.094392 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3bdc0c-9c7c-4b90-804f-36870230590e admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:24.098702 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:24.155312 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d907fc97-1d59-420c-b82a-138feb6c16d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:24.155962 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907fc97-1d59-420c-b82a-138feb6c16d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:24.156283 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907fc97-1d59-420c-b82a-138feb6c16d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:24.156510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907fc97-1d59-420c-b82a-138feb6c16d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:24.157166 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d907fc97-1d59-420c-b82a-138feb6c16d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:24.159241 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d907fc97-1d59-420c-b82a-138feb6c16d8 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:24.169342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1105/2176] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:21:24.173630 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f812b1e-f1cc-43ca-a9c9-aaf12e848e71 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:24.184446 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f812b1e-f1cc-43ca-a9c9-aaf12e848e71 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:24.189135 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f299d7-c864-4059-8bf5-74b06fa7039d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:24.189967 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f299d7-c864-4059-8bf5-74b06fa7039d admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:24.190247 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f299d7-c864-4059-8bf5-74b06fa7039d admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:24.190520 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f299d7-c864-4059-8bf5-74b06fa7039d admin admin] PATH_INFO: `/v3/projects/606b96a009524ac0908d37fce333a84f/users/217bce795ae04ee280071224db6d26d4/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:24.192607 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f299d7-c864-4059-8bf5-74b06fa7039d admin admin] RBAC: Authorizing `identity:create_grant(project_id=606b96a009524ac0908d37fce333a84f, user_id=217bce795ae04ee280071224db6d26d4, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:24.195245 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f299d7-c864-4059-8bf5-74b06fa7039d admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:24.205636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1072/2177] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:21:24 2022] PUT /identity/v3/projects/606b96a009524ac0908d37fce333a84f/users/217bce795ae04ee280071224db6d26d4/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:24.216748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63ebb1-59ae-4d63-bf56-8944b31131f6 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:24.217035 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63ebb1-59ae-4d63-bf56-8944b31131f6 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:24.217313 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63ebb1-59ae-4d63-bf56-8944b31131f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:24.246397 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-0c63ebb1-59ae-4d63-bf56-8944b31131f6 None None] MFA Rules not processed for user `217bce795ae04ee280071224db6d26d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:24.247633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-0c63ebb1-59ae-4d63-bf56-8944b31131f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:24.317985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1106/2178] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:24 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:24.329631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f299d7-c864-4059-8bf5-74b06fa7039d admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:24.331954 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-27f299d7-c864-4059-8bf5-74b06fa7039d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:24.402230 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:24.406633 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f299d7-c864-4059-8bf5-74b06fa7039d admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:24.412005 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa32a9c-26d9-4b9e-b383-ac4c48daabd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:24.412914 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa32a9c-26d9-4b9e-b383-ac4c48daabd7 service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:24.413194 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa32a9c-26d9-4b9e-b383-ac4c48daabd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:24.413470 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa32a9c-26d9-4b9e-b383-ac4c48daabd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:24.479074 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa32a9c-26d9-4b9e-b383-ac4c48daabd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:24.480550 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa32a9c-26d9-4b9e-b383-ac4c48daabd7 service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:24.489832 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1073/2179] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:21:26.453465 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63ebb1-59ae-4d63-bf56-8944b31131f6 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:26.454921 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-0c63ebb1-59ae-4d63-bf56-8944b31131f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:26.522287 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63ebb1-59ae-4d63-bf56-8944b31131f6 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:26.528276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb5054-d2c9-4f09-94dd-8285f8c7b12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:26.529264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb5054-d2c9-4f09-94dd-8285f8c7b12f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:26.529757 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb5054-d2c9-4f09-94dd-8285f8c7b12f admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:26.530217 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb5054-d2c9-4f09-94dd-8285f8c7b12f admin admin] PATH_INFO: `/v3/users/7d82ba9bce3546e682bcacdcd8fb1ed5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:26.532044 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb5054-d2c9-4f09-94dd-8285f8c7b12f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d82ba9bce3546e682bcacdcd8fb1ed5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:26.535503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb5054-d2c9-4f09-94dd-8285f8c7b12f admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:26.576082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:26.594099 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-43cb5054-d2c9-4f09-94dd-8285f8c7b12f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d82ba9bce3546e682bcacdcd8fb1ed5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:26.602963 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-43cb5054-d2c9-4f09-94dd-8285f8c7b12f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d82ba9bce3546e682bcacdcd8fb1ed5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:26.603809 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-43cb5054-d2c9-4f09-94dd-8285f8c7b12f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d82ba9bce3546e682bcacdcd8fb1ed5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:26.605103 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-43cb5054-d2c9-4f09-94dd-8285f8c7b12f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d82ba9bce3546e682bcacdcd8fb1ed5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:26.627780 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-43cb5054-d2c9-4f09-94dd-8285f8c7b12f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d82ba9bce3546e682bcacdcd8fb1ed5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:26.639036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1107/2180] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:21:26 2022] DELETE /identity/v3/users/7d82ba9bce3546e682bcacdcd8fb1ed5 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:21:26.640843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35b4c9-be19-48a2-884f-0e690b18c7eb None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:26.641300 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35b4c9-be19-48a2-884f-0e690b18c7eb None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:26.641394 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35b4c9-be19-48a2-884f-0e690b18c7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:26.672996 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-2e35b4c9-be19-48a2-884f-0e690b18c7eb None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:26.674757 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-2e35b4c9-be19-48a2-884f-0e690b18c7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:26.751743 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1074/2181] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:26.760984 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb5054-d2c9-4f09-94dd-8285f8c7b12f admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:26.762423 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-43cb5054-d2c9-4f09-94dd-8285f8c7b12f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:26.832380 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb5054-d2c9-4f09-94dd-8285f8c7b12f admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:26.836717 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a98626-cf2c-4562-9154-af6093338168 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:26.837382 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a98626-cf2c-4562-9154-af6093338168 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:26.837664 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a98626-cf2c-4562-9154-af6093338168 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:26.838039 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a98626-cf2c-4562-9154-af6093338168 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:26.853861 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a98626-cf2c-4562-9154-af6093338168 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:26.855724 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a98626-cf2c-4562-9154-af6093338168 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:26.868835 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1108/2182] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:21:27.276962 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35b4c9-be19-48a2-884f-0e690b18c7eb None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:27.278716 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-2e35b4c9-be19-48a2-884f-0e690b18c7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:27.322860 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a98626-cf2c-4562-9154-af6093338168 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:27.325195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-09a98626-cf2c-4562-9154-af6093338168 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:27.340342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a98626-cf2c-4562-9154-af6093338168 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:27.346228 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44f9c0-6544-4b2e-9b7c-dee951701e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:27.347507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44f9c0-6544-4b2e-9b7c-dee951701e82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:27.347507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44f9c0-6544-4b2e-9b7c-dee951701e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:27.347707 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44f9c0-6544-4b2e-9b7c-dee951701e82 admin admin] PATH_INFO: `/v3/projects/a0e1ac9f3a274ba2b2469dbdc2ebf2e5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:27.348761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed44f9c0-6544-4b2e-9b7c-dee951701e82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0e1ac9f3a274ba2b2469dbdc2ebf2e5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:27.350735 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed44f9c0-6544-4b2e-9b7c-dee951701e82 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:27.397198 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35b4c9-be19-48a2-884f-0e690b18c7eb None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:27.401295 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc81d39-de0e-4ee3-81e0-c9b4c798f5cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:27.402180 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc81d39-de0e-4ee3-81e0-c9b4c798f5cb service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:27.402320 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc81d39-de0e-4ee3-81e0-c9b4c798f5cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:27.402500 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc81d39-de0e-4ee3-81e0-c9b4c798f5cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:27.405232 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-9bc81d39-de0e-4ee3-81e0-c9b4c798f5cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:27.440489 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ed44f9c0-6544-4b2e-9b7c-dee951701e82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0e1ac9f3a274ba2b2469dbdc2ebf2e5', 'request_id': 'req-ed44f9c0-6544-4b2e-9b7c-dee951701e82'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:27.446376 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ed44f9c0-6544-4b2e-9b7c-dee951701e82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0e1ac9f3a274ba2b2469dbdc2ebf2e5', 'request_id': 'req-ed44f9c0-6544-4b2e-9b7c-dee951701e82'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:27.454260 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ed44f9c0-6544-4b2e-9b7c-dee951701e82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0e1ac9f3a274ba2b2469dbdc2ebf2e5', 'request_id': 'req-ed44f9c0-6544-4b2e-9b7c-dee951701e82'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:27.469237 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ed44f9c0-6544-4b2e-9b7c-dee951701e82 admin admin] The token cache is being invalidate because project a0e1ac9f3a274ba2b2469dbdc2ebf2e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:21:27.469519 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ed44f9c0-6544-4b2e-9b7c-dee951701e82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:27.470364 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ed44f9c0-6544-4b2e-9b7c-dee951701e82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:27.472637 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1109/2183] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:21:27 2022] DELETE /identity/v3/projects/a0e1ac9f3a274ba2b2469dbdc2ebf2e5 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:27.476909 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44f9c0-6544-4b2e-9b7c-dee951701e82 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:27.478852 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ed44f9c0-6544-4b2e-9b7c-dee951701e82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:27.487891 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc81d39-de0e-4ee3-81e0-c9b4c798f5cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:27.489352 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc81d39-de0e-4ee3-81e0-c9b4c798f5cb service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:27.493597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:27.502960 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1075/2184] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:21:27.535696 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:27.575088 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44f9c0-6544-4b2e-9b7c-dee951701e82 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:27.581145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c97b76-5404-455b-8e9f-7c124ed49a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:27.581795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c97b76-5404-455b-8e9f-7c124ed49a67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:27.582153 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c97b76-5404-455b-8e9f-7c124ed49a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:27.582354 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c97b76-5404-455b-8e9f-7c124ed49a67 admin admin] PATH_INFO: `/v3/users/c529b3d9b76549cb90eac41b3cfec6a9` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:27.583634 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c97b76-5404-455b-8e9f-7c124ed49a67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c529b3d9b76549cb90eac41b3cfec6a9)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:27.585697 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c97b76-5404-455b-8e9f-7c124ed49a67 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:27.642058 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c4c97b76-5404-455b-8e9f-7c124ed49a67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c529b3d9b76549cb90eac41b3cfec6a9'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:27.650464 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c4c97b76-5404-455b-8e9f-7c124ed49a67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c529b3d9b76549cb90eac41b3cfec6a9'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:27.651358 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c4c97b76-5404-455b-8e9f-7c124ed49a67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c529b3d9b76549cb90eac41b3cfec6a9'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:27.652220 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c4c97b76-5404-455b-8e9f-7c124ed49a67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c529b3d9b76549cb90eac41b3cfec6a9'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:27.667917 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c4c97b76-5404-455b-8e9f-7c124ed49a67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c529b3d9b76549cb90eac41b3cfec6a9'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:27.682905 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1110/2185] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:21:27 2022] DELETE /identity/v3/users/c529b3d9b76549cb90eac41b3cfec6a9 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:21:28.035413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc81d39-de0e-4ee3-81e0-c9b4c798f5cb service cinder] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:28.047374 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc81d39-de0e-4ee3-81e0-c9b4c798f5cb service cinder] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:28.119281 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece8c9a2-cdc0-41d9-a87b-de540ac1a8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:28.119980 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8c9a2-cdc0-41d9-a87b-de540ac1a8cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:28.120265 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8c9a2-cdc0-41d9-a87b-de540ac1a8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:28.120541 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8c9a2-cdc0-41d9-a87b-de540ac1a8cf admin admin] PATH_INFO: `/v3/projects/e8e8bb26775044aea7ee69c764dccf15` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:28.121719 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece8c9a2-cdc0-41d9-a87b-de540ac1a8cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8e8bb26775044aea7ee69c764dccf15)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:28.123713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece8c9a2-cdc0-41d9-a87b-de540ac1a8cf admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:28.180856 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c97b76-5404-455b-8e9f-7c124ed49a67 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:28.182279 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c4c97b76-5404-455b-8e9f-7c124ed49a67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:28.185893 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-ece8c9a2-cdc0-41d9-a87b-de540ac1a8cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8e8bb26775044aea7ee69c764dccf15', 'request_id': 'req-ece8c9a2-cdc0-41d9-a87b-de540ac1a8cf'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:28.196768 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-ece8c9a2-cdc0-41d9-a87b-de540ac1a8cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8e8bb26775044aea7ee69c764dccf15', 'request_id': 'req-ece8c9a2-cdc0-41d9-a87b-de540ac1a8cf'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:28.203600 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-ece8c9a2-cdc0-41d9-a87b-de540ac1a8cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8e8bb26775044aea7ee69c764dccf15', 'request_id': 'req-ece8c9a2-cdc0-41d9-a87b-de540ac1a8cf'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:28.223998 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-ece8c9a2-cdc0-41d9-a87b-de540ac1a8cf admin admin] The token cache is being invalidate because project e8e8bb26775044aea7ee69c764dccf15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:21:28.224340 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-ece8c9a2-cdc0-41d9-a87b-de540ac1a8cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:28.227055 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-ece8c9a2-cdc0-41d9-a87b-de540ac1a8cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:28.230038 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1076/2186] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:21:28 2022] DELETE /identity/v3/projects/e8e8bb26775044aea7ee69c764dccf15 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:28.235572 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73ceae-c858-49f0-87f0-8fd6531fea8c None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:28.235970 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73ceae-c858-49f0-87f0-8fd6531fea8c None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:28.236842 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73ceae-c858-49f0-87f0-8fd6531fea8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:28.261919 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c97b76-5404-455b-8e9f-7c124ed49a67 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:28.265092 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-9c73ceae-c858-49f0-87f0-8fd6531fea8c None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:28.266401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-9c73ceae-c858-49f0-87f0-8fd6531fea8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:28.329319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765b221-ab8d-4b71-b803-f5ebaecc5e30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:28.330088 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765b221-ab8d-4b71-b803-f5ebaecc5e30 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:28.332081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765b221-ab8d-4b71-b803-f5ebaecc5e30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:28.332986 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765b221-ab8d-4b71-b803-f5ebaecc5e30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:28.336224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f765b221-ab8d-4b71-b803-f5ebaecc5e30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:28.367338 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1077/2187] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:21:28.373717 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f712a9-7bc7-43c9-9a9b-053d4a2af03c None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:28.374134 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f712a9-7bc7-43c9-9a9b-053d4a2af03c None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:28.375584 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f712a9-7bc7-43c9-9a9b-053d4a2af03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:28.382806 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:28.400522 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-c2f712a9-7bc7-43c9-9a9b-053d4a2af03c None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:28.402733 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-c2f712a9-7bc7-43c9-9a9b-053d4a2af03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:28.416352 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1078/2188] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:28.420652 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f712a9-7bc7-43c9-9a9b-053d4a2af03c None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:28.430860 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f712a9-7bc7-43c9-9a9b-053d4a2af03c None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:28.444319 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cdb3ad-e31d-450c-8544-7d6842eb254d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:28.444856 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cdb3ad-e31d-450c-8544-7d6842eb254d admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:28.445163 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cdb3ad-e31d-450c-8544-7d6842eb254d admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:28.445405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cdb3ad-e31d-450c-8544-7d6842eb254d admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:28.446074 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cdb3ad-e31d-450c-8544-7d6842eb254d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:21:28.446382 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cdb3ad-e31d-450c-8544-7d6842eb254d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:28.451515 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765b221-ab8d-4b71-b803-f5ebaecc5e30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:28.452240 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cdb3ad-e31d-450c-8544-7d6842eb254d admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:28.453195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765b221-ab8d-4b71-b803-f5ebaecc5e30 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:28.464808 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1111/2189] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:28 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:21:28.472215 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1079/2190] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:21:28.476977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765b221-ab8d-4b71-b803-f5ebaecc5e30 service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:28.488959 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765b221-ab8d-4b71-b803-f5ebaecc5e30 service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:28.499729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61a096-2057-4371-86c0-291dd1824695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:28.499729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61a096-2057-4371-86c0-291dd1824695 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:28.499729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61a096-2057-4371-86c0-291dd1824695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:28.499729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61a096-2057-4371-86c0-291dd1824695 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:28.500056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d61a096-2057-4371-86c0-291dd1824695 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:28.500056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d61a096-2057-4371-86c0-291dd1824695 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:28.536401 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1112/2191] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:21:28 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:21:28.540417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cdb3ad-e31d-450c-8544-7d6842eb254d admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:28.554925 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cdb3ad-e31d-450c-8544-7d6842eb254d admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:28.561349 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:28.643978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18600f2e-498f-4ae6-8f11-7c93beac1be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:28.644620 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18600f2e-498f-4ae6-8f11-7c93beac1be9 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:28.644815 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18600f2e-498f-4ae6-8f11-7c93beac1be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:28.645134 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18600f2e-498f-4ae6-8f11-7c93beac1be9 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:28.645852 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18600f2e-498f-4ae6-8f11-7c93beac1be9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:28.648029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18600f2e-498f-4ae6-8f11-7c93beac1be9 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:28.681180 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1080/2192] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:21:28 2022] POST /identity/v3/users => generated 294 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:21:28.682538 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61a096-2057-4371-86c0-291dd1824695 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:28.697894 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61a096-2057-4371-86c0-291dd1824695 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:28.703600 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904c3080-b6ca-47ec-a95e-3bcd2adc3662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:28.704305 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c3080-b6ca-47ec-a95e-3bcd2adc3662 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:28.704699 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c3080-b6ca-47ec-a95e-3bcd2adc3662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:28.705097 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c3080-b6ca-47ec-a95e-3bcd2adc3662 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:28.705916 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904c3080-b6ca-47ec-a95e-3bcd2adc3662 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:28.707653 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904c3080-b6ca-47ec-a95e-3bcd2adc3662 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:28.722022 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1113/2193] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:21:28.725823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18600f2e-498f-4ae6-8f11-7c93beac1be9 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:28.737737 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18600f2e-498f-4ae6-8f11-7c93beac1be9 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:28.745294 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a77bd-ce91-4770-b4b1-d86d3711d133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:28.745901 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a77bd-ce91-4770-b4b1-d86d3711d133 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:28.746169 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a77bd-ce91-4770-b4b1-d86d3711d133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:28.746788 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a77bd-ce91-4770-b4b1-d86d3711d133 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:28.747147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a77bd-ce91-4770-b4b1-d86d3711d133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:28.748795 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a77bd-ce91-4770-b4b1-d86d3711d133 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:28.763441 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1081/2194] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:21:28.768734 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904c3080-b6ca-47ec-a95e-3bcd2adc3662 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:28.782323 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904c3080-b6ca-47ec-a95e-3bcd2adc3662 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:28.787776 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5480e65-72ff-44d8-a7c0-0b08834e4530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:28.789173 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5480e65-72ff-44d8-a7c0-0b08834e4530 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:28.789616 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5480e65-72ff-44d8-a7c0-0b08834e4530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:28.790038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5480e65-72ff-44d8-a7c0-0b08834e4530 admin admin] PATH_INFO: `/v3/projects/c19c074002b040b188082b3d1f115351/users/02dae8a643864d0f8fa78c0677c31126/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:28.810969 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5480e65-72ff-44d8-a7c0-0b08834e4530 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c19c074002b040b188082b3d1f115351, user_id=02dae8a643864d0f8fa78c0677c31126, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:28.814762 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5480e65-72ff-44d8-a7c0-0b08834e4530 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:28.834148 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1114/2195] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:21:28 2022] PUT /identity/v3/projects/c19c074002b040b188082b3d1f115351/users/02dae8a643864d0f8fa78c0677c31126/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:28.844878 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d31d40-3bce-468c-a440-d6e03e1bb56f None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:28.846286 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d31d40-3bce-468c-a440-d6e03e1bb56f None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:28.846753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d31d40-3bce-468c-a440-d6e03e1bb56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:28.888467 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-56d31d40-3bce-468c-a440-d6e03e1bb56f None None] MFA Rules not processed for user `02dae8a643864d0f8fa78c0677c31126`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:28.889893 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-56d31d40-3bce-468c-a440-d6e03e1bb56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:29.000486 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1082/2196] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:21:29.007600 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5480e65-72ff-44d8-a7c0-0b08834e4530 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:29.085348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5480e65-72ff-44d8-a7c0-0b08834e4530 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:29.094139 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122312e0-6ab5-4f34-824a-7b776b0cd4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:29.095295 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122312e0-6ab5-4f34-824a-7b776b0cd4f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:29.096341 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122312e0-6ab5-4f34-824a-7b776b0cd4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:29.096813 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122312e0-6ab5-4f34-824a-7b776b0cd4f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:29.098614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122312e0-6ab5-4f34-824a-7b776b0cd4f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:29.101237 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122312e0-6ab5-4f34-824a-7b776b0cd4f4 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:29.141826 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1115/2197] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:21:29 2022] POST /identity/v3/projects => generated 387 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:21:29.146031 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d31d40-3bce-468c-a440-d6e03e1bb56f None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:29.164644 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d31d40-3bce-468c-a440-d6e03e1bb56f None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:29.233061 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f44744-6d1c-4f0e-a460-1392d9e9712d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:29.233688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f44744-6d1c-4f0e-a460-1392d9e9712d admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:29.233983 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f44744-6d1c-4f0e-a460-1392d9e9712d admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:29.234966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f44744-6d1c-4f0e-a460-1392d9e9712d admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:29.235675 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f44744-6d1c-4f0e-a460-1392d9e9712d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:29.238270 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f44744-6d1c-4f0e-a460-1392d9e9712d admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:29.263120 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1083/2198] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:21:29 2022] POST /identity/v3/users => generated 300 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:21:29.267409 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122312e0-6ab5-4f34-824a-7b776b0cd4f4 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:29.281108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122312e0-6ab5-4f34-824a-7b776b0cd4f4 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:29.286822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e1323-5c61-4e70-b944-59142c973b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:29.287455 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e1323-5c61-4e70-b944-59142c973b77 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:29.287767 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e1323-5c61-4e70-b944-59142c973b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:29.288051 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e1323-5c61-4e70-b944-59142c973b77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:29.288712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649e1323-5c61-4e70-b944-59142c973b77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:29.290346 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649e1323-5c61-4e70-b944-59142c973b77 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:29.311122 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1116/2199] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:21:29.315158 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f44744-6d1c-4f0e-a460-1392d9e9712d admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:29.325936 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f44744-6d1c-4f0e-a460-1392d9e9712d admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:29.331055 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf161c-5bd1-4ddb-af14-ddf0754712be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:29.331766 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf161c-5bd1-4ddb-af14-ddf0754712be admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:29.332116 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf161c-5bd1-4ddb-af14-ddf0754712be admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:29.332784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf161c-5bd1-4ddb-af14-ddf0754712be admin admin] PATH_INFO: `/v3/projects/09d0fcf55384428d9d6b41047194a303/users/5375dcbca25d4515b2711c30ca1a54f3/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:29.335767 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:29.351737 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf161c-5bd1-4ddb-af14-ddf0754712be admin admin] RBAC: Authorizing `identity:create_grant(project_id=09d0fcf55384428d9d6b41047194a303, user_id=5375dcbca25d4515b2711c30ca1a54f3, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:29.354421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf161c-5bd1-4ddb-af14-ddf0754712be admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:29.371787 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1084/2200] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:21:29 2022] PUT /identity/v3/projects/09d0fcf55384428d9d6b41047194a303/users/5375dcbca25d4515b2711c30ca1a54f3/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:29.375238 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e1323-5c61-4e70-b944-59142c973b77 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:29.387025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e1323-5c61-4e70-b944-59142c973b77 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:29.427734 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:29.460405 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afd6c1-cff8-4067-86a7-74db72c1fad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:29.461341 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afd6c1-cff8-4067-86a7-74db72c1fad6 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:29.461460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afd6c1-cff8-4067-86a7-74db72c1fad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:29.461794 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afd6c1-cff8-4067-86a7-74db72c1fad6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:29.462624 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46afd6c1-cff8-4067-86a7-74db72c1fad6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:29.464487 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46afd6c1-cff8-4067-86a7-74db72c1fad6 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:29.487935 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1117/2201] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:21:29.488576 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf161c-5bd1-4ddb-af14-ddf0754712be admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:29.493679 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf161c-5bd1-4ddb-af14-ddf0754712be admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:29.499834 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f23b8-b6ee-47c5-af72-1fa5b8897c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:29.500624 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f23b8-b6ee-47c5-af72-1fa5b8897c61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:29.500906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f23b8-b6ee-47c5-af72-1fa5b8897c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:29.501181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f23b8-b6ee-47c5-af72-1fa5b8897c61 admin admin] PATH_INFO: `/v3/projects/09d0fcf55384428d9d6b41047194a303/users/5375dcbca25d4515b2711c30ca1a54f3/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:29.503635 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f23b8-b6ee-47c5-af72-1fa5b8897c61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09d0fcf55384428d9d6b41047194a303, user_id=5375dcbca25d4515b2711c30ca1a54f3, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:29.507812 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f23b8-b6ee-47c5-af72-1fa5b8897c61 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:29.519614 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1085/2202] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:21:29 2022] PUT /identity/v3/projects/09d0fcf55384428d9d6b41047194a303/users/5375dcbca25d4515b2711c30ca1a54f3/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:21:29.531393 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffb559-339e-41cd-936a-be6d7e534dcb None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:29.531726 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffb559-339e-41cd-936a-be6d7e534dcb None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:29.532167 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffb559-339e-41cd-936a-be6d7e534dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:29.566474 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-95ffb559-339e-41cd-936a-be6d7e534dcb None None] MFA Rules not processed for user `5375dcbca25d4515b2711c30ca1a54f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:29.567786 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-95ffb559-339e-41cd-936a-be6d7e534dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:29.591950 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f23b8-b6ee-47c5-af72-1fa5b8897c61 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:29.591950 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-ca4f23b8-b6ee-47c5-af72-1fa5b8897c61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:29.691800 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1118/2203] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:29 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:29.707925 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f23b8-b6ee-47c5-af72-1fa5b8897c61 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:29.715092 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ffb559-339e-41cd-936a-be6d7e534dcb None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:29.715092 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-95ffb559-339e-41cd-936a-be6d7e534dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:29.717204 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c25272-24c1-4bf9-aa18-8b6899beeb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:29.717856 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c25272-24c1-4bf9-aa18-8b6899beeb41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:29.718132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c25272-24c1-4bf9-aa18-8b6899beeb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:29.718408 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c25272-24c1-4bf9-aa18-8b6899beeb41 admin admin] PATH_INFO: `/v3/users/d26f8c676bc549eeb2a3b54cd577f95c` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:29.722869 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c25272-24c1-4bf9-aa18-8b6899beeb41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d26f8c676bc549eeb2a3b54cd577f95c)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:29.730295 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c25272-24c1-4bf9-aa18-8b6899beeb41 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:29.803745 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-16c25272-24c1-4bf9-aa18-8b6899beeb41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd26f8c676bc549eeb2a3b54cd577f95c'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:29.813956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-16c25272-24c1-4bf9-aa18-8b6899beeb41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd26f8c676bc549eeb2a3b54cd577f95c'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:29.815432 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-16c25272-24c1-4bf9-aa18-8b6899beeb41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd26f8c676bc549eeb2a3b54cd577f95c'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:29.821224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ffb559-339e-41cd-936a-be6d7e534dcb None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:29.827505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5bb149-a466-49ca-9d21-57f7cd61a870 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:29.828446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5bb149-a466-49ca-9d21-57f7cd61a870 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:29.829173 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5bb149-a466-49ca-9d21-57f7cd61a870 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:29.829349 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5bb149-a466-49ca-9d21-57f7cd61a870 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:29.851474 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5bb149-a466-49ca-9d21-57f7cd61a870 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:29.851474 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5bb149-a466-49ca-9d21-57f7cd61a870 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:29.851858 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-16c25272-24c1-4bf9-aa18-8b6899beeb41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd26f8c676bc549eeb2a3b54cd577f95c'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:29.855358 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1119/2204] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:29 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:21:29.858069 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-16c25272-24c1-4bf9-aa18-8b6899beeb41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd26f8c676bc549eeb2a3b54cd577f95c'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:29.871569 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1086/2205] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:21:29 2022] DELETE /identity/v3/users/d26f8c676bc549eeb2a3b54cd577f95c => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:29.873457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd512f3-1c9e-495b-9dec-6c87b5c1bd61 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:29.873802 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd512f3-1c9e-495b-9dec-6c87b5c1bd61 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:29.874084 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd512f3-1c9e-495b-9dec-6c87b5c1bd61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:29.897948 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-8bd512f3-1c9e-495b-9dec-6c87b5c1bd61 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:29.899751 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8bd512f3-1c9e-495b-9dec-6c87b5c1bd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:29.900759 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c25272-24c1-4bf9-aa18-8b6899beeb41 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:29.904287 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-16c25272-24c1-4bf9-aa18-8b6899beeb41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:29.990469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:30.007262 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1120/2206] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:30.018966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:30.022293 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd512f3-1c9e-495b-9dec-6c87b5c1bd61 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:30.029820 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8bd512f3-1c9e-495b-9dec-6c87b5c1bd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:30.067817 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c25272-24c1-4bf9-aa18-8b6899beeb41 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:30.072902 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b6387-75ed-46fe-aef6-66b572b6f4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:30.073549 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b6387-75ed-46fe-aef6-66b572b6f4c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:30.073832 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b6387-75ed-46fe-aef6-66b572b6f4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:30.074105 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b6387-75ed-46fe-aef6-66b572b6f4c0 admin admin] PATH_INFO: `/v3/users/02dae8a643864d0f8fa78c0677c31126` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:30.075254 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b6387-75ed-46fe-aef6-66b572b6f4c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02dae8a643864d0f8fa78c0677c31126)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:30.077220 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b6387-75ed-46fe-aef6-66b572b6f4c0 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:30.143213 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd512f3-1c9e-495b-9dec-6c87b5c1bd61 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:30.156351 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48e74e-3813-434c-942e-f09c10c3ef3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:30.157702 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48e74e-3813-434c-942e-f09c10c3ef3e service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:30.158921 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48e74e-3813-434c-942e-f09c10c3ef3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:30.159466 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48e74e-3813-434c-942e-f09c10c3ef3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:30.177803 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca48e74e-3813-434c-942e-f09c10c3ef3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:30.179268 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca48e74e-3813-434c-942e-f09c10c3ef3e service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:30.186688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-137b6387-75ed-46fe-aef6-66b572b6f4c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02dae8a643864d0f8fa78c0677c31126'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:30.190198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1121/2207] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:21:30.207893 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-137b6387-75ed-46fe-aef6-66b572b6f4c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02dae8a643864d0f8fa78c0677c31126'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:30.210002 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-137b6387-75ed-46fe-aef6-66b572b6f4c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02dae8a643864d0f8fa78c0677c31126'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:30.227016 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-137b6387-75ed-46fe-aef6-66b572b6f4c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02dae8a643864d0f8fa78c0677c31126'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:30.235789 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-137b6387-75ed-46fe-aef6-66b572b6f4c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02dae8a643864d0f8fa78c0677c31126'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:30.243553 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1087/2208] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:21:29 2022] DELETE /identity/v3/users/02dae8a643864d0f8fa78c0677c31126 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:30.247651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc067876-6628-4d4b-a17a-52640177144f None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:30.247945 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc067876-6628-4d4b-a17a-52640177144f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:30.248237 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc067876-6628-4d4b-a17a-52640177144f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:30.268329 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-fc067876-6628-4d4b-a17a-52640177144f None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:30.269544 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-fc067876-6628-4d4b-a17a-52640177144f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:30.367473 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1122/2209] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:21:30.378998 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b6387-75ed-46fe-aef6-66b572b6f4c0 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:30.381383 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-137b6387-75ed-46fe-aef6-66b572b6f4c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:30.474017 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b6387-75ed-46fe-aef6-66b572b6f4c0 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:30.479013 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756491da-7360-4e06-a2bc-e049ed8a4a21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:30.479847 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756491da-7360-4e06-a2bc-e049ed8a4a21 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:30.480043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756491da-7360-4e06-a2bc-e049ed8a4a21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:30.480291 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756491da-7360-4e06-a2bc-e049ed8a4a21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:30.492138 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756491da-7360-4e06-a2bc-e049ed8a4a21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:30.493322 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756491da-7360-4e06-a2bc-e049ed8a4a21 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:30.506639 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1088/2210] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:21:30.596496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc067876-6628-4d4b-a17a-52640177144f None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:30.598334 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-fc067876-6628-4d4b-a17a-52640177144f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:30.611842 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc067876-6628-4d4b-a17a-52640177144f None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:30.624896 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789727c-0e20-4c29-adf3-8fe1b3b92358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:30.626151 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789727c-0e20-4c29-adf3-8fe1b3b92358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:30.626424 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789727c-0e20-4c29-adf3-8fe1b3b92358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:30.627117 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789727c-0e20-4c29-adf3-8fe1b3b92358 admin admin] PATH_INFO: `/v3/projects/296bd099e9f8492aaeae70cf38f969c9` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:30.631670 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7789727c-0e20-4c29-adf3-8fe1b3b92358 admin admin] RBAC: Authorizing `identity:delete_project(project_id=296bd099e9f8492aaeae70cf38f969c9)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:30.639368 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7789727c-0e20-4c29-adf3-8fe1b3b92358 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:30.754476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7789727c-0e20-4c29-adf3-8fe1b3b92358 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '296bd099e9f8492aaeae70cf38f969c9', 'request_id': 'req-7789727c-0e20-4c29-adf3-8fe1b3b92358'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:30.762752 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7789727c-0e20-4c29-adf3-8fe1b3b92358 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '296bd099e9f8492aaeae70cf38f969c9', 'request_id': 'req-7789727c-0e20-4c29-adf3-8fe1b3b92358'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:30.778868 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7789727c-0e20-4c29-adf3-8fe1b3b92358 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '296bd099e9f8492aaeae70cf38f969c9', 'request_id': 'req-7789727c-0e20-4c29-adf3-8fe1b3b92358'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:30.797580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7789727c-0e20-4c29-adf3-8fe1b3b92358 admin admin] The token cache is being invalidate because project 296bd099e9f8492aaeae70cf38f969c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:21:30.797875 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7789727c-0e20-4c29-adf3-8fe1b3b92358 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:30.799589 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7789727c-0e20-4c29-adf3-8fe1b3b92358 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:30.801411 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1123/2211] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:21:30 2022] DELETE /identity/v3/projects/296bd099e9f8492aaeae70cf38f969c9 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:30.806415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4870f-9184-4584-969b-dcd577bd8464 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:30.806696 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4870f-9184-4584-969b-dcd577bd8464 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:30.806966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4870f-9184-4584-969b-dcd577bd8464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:30.827738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-64a4870f-9184-4584-969b-dcd577bd8464 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:30.828946 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-64a4870f-9184-4584-969b-dcd577bd8464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:30.940174 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789727c-0e20-4c29-adf3-8fe1b3b92358 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:30.941512 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1089/2212] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:30.945104 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7789727c-0e20-4c29-adf3-8fe1b3b92358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:30.950617 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018ef52-1c60-4a51-b748-fe68df5e4f23 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:30.951725 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018ef52-1c60-4a51-b748-fe68df5e4f23 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:30.952626 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018ef52-1c60-4a51-b748-fe68df5e4f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:30.960041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789727c-0e20-4c29-adf3-8fe1b3b92358 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:30.965896 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17e4d0-ebc6-4ffe-92d5-e52b82d1841d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:30.966609 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17e4d0-ebc6-4ffe-92d5-e52b82d1841d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:30.966971 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17e4d0-ebc6-4ffe-92d5-e52b82d1841d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:30.967297 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17e4d0-ebc6-4ffe-92d5-e52b82d1841d admin admin] PATH_INFO: `/v3/projects/c19c074002b040b188082b3d1f115351` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:30.969764 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17e4d0-ebc6-4ffe-92d5-e52b82d1841d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c19c074002b040b188082b3d1f115351)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:30.972172 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17e4d0-ebc6-4ffe-92d5-e52b82d1841d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:30.976797 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-9018ef52-1c60-4a51-b748-fe68df5e4f23 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:30.978002 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-9018ef52-1c60-4a51-b748-fe68df5e4f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:30.986903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1090/2213] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:30.990873 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018ef52-1c60-4a51-b748-fe68df5e4f23 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:30.996550 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:31.001613 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018ef52-1c60-4a51-b748-fe68df5e4f23 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:31.016372 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516afa3-1a31-4505-944d-4d0e25531649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:31.016995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516afa3-1a31-4505-944d-4d0e25531649 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:31.017373 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516afa3-1a31-4505-944d-4d0e25531649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:31.018138 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516afa3-1a31-4505-944d-4d0e25531649 admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:31.018827 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e516afa3-1a31-4505-944d-4d0e25531649 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:21:31.019130 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e516afa3-1a31-4505-944d-4d0e25531649 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:31.020778 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e516afa3-1a31-4505-944d-4d0e25531649 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:31.021201 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:31.038707 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1091/2214] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:21:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:21:31.043071 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516afa3-1a31-4505-944d-4d0e25531649 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:31.071133 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-be17e4d0-ebc6-4ffe-92d5-e52b82d1841d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c19c074002b040b188082b3d1f115351', 'request_id': 'req-be17e4d0-ebc6-4ffe-92d5-e52b82d1841d'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:31.073365 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516afa3-1a31-4505-944d-4d0e25531649 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:31.081680 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-be17e4d0-ebc6-4ffe-92d5-e52b82d1841d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c19c074002b040b188082b3d1f115351', 'request_id': 'req-be17e4d0-ebc6-4ffe-92d5-e52b82d1841d'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:31.090619 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-be17e4d0-ebc6-4ffe-92d5-e52b82d1841d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c19c074002b040b188082b3d1f115351', 'request_id': 'req-be17e4d0-ebc6-4ffe-92d5-e52b82d1841d'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:31.117731 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-be17e4d0-ebc6-4ffe-92d5-e52b82d1841d admin admin] The token cache is being invalidate because project c19c074002b040b188082b3d1f115351 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:21:31.123921 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-be17e4d0-ebc6-4ffe-92d5-e52b82d1841d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:31.124914 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-be17e4d0-ebc6-4ffe-92d5-e52b82d1841d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:31.126555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1124/2215] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:21:30 2022] DELETE /identity/v3/projects/c19c074002b040b188082b3d1f115351 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:21:31.132084 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17e4d0-ebc6-4ffe-92d5-e52b82d1841d admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:31.133750 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-be17e4d0-ebc6-4ffe-92d5-e52b82d1841d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:31.170116 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c4d91-4fd1-4476-934f-79924b8e2d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:31.172645 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c4d91-4fd1-4476-934f-79924b8e2d1b admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:31.173190 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c4d91-4fd1-4476-934f-79924b8e2d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:31.174209 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c4d91-4fd1-4476-934f-79924b8e2d1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:31.175241 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c4d91-4fd1-4476-934f-79924b8e2d1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:31.178641 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c4d91-4fd1-4476-934f-79924b8e2d1b admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:31.219500 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1092/2216] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:21:31 2022] POST /identity/v3/projects => generated 373 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:21:31.223969 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c4d91-4fd1-4476-934f-79924b8e2d1b admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:31.225836 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-157c4d91-4fd1-4476-934f-79924b8e2d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:31.226154 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17e4d0-ebc6-4ffe-92d5-e52b82d1841d admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:31.301237 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0039a-ac8b-48d0-ba0c-a9b70e02e95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:31.301886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0039a-ac8b-48d0-ba0c-a9b70e02e95e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:31.302169 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0039a-ac8b-48d0-ba0c-a9b70e02e95e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:31.302472 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0039a-ac8b-48d0-ba0c-a9b70e02e95e admin admin] PATH_INFO: `/v3/users/5375dcbca25d4515b2711c30ca1a54f3` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:31.303679 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b0039a-ac8b-48d0-ba0c-a9b70e02e95e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5375dcbca25d4515b2711c30ca1a54f3)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:31.305622 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b0039a-ac8b-48d0-ba0c-a9b70e02e95e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:31.314931 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c4d91-4fd1-4476-934f-79924b8e2d1b admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:31.320471 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad971f89-3e0c-4056-bf53-b84bd9e43b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:31.321556 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad971f89-3e0c-4056-bf53-b84bd9e43b13 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:31.321843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad971f89-3e0c-4056-bf53-b84bd9e43b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:31.322119 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad971f89-3e0c-4056-bf53-b84bd9e43b13 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:31.322771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad971f89-3e0c-4056-bf53-b84bd9e43b13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:31.324745 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad971f89-3e0c-4056-bf53-b84bd9e43b13 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:31.352933 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1093/2217] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:21:31 2022] POST /identity/v3/users => generated 287 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:21:31.357365 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad971f89-3e0c-4056-bf53-b84bd9e43b13 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:31.373568 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad971f89-3e0c-4056-bf53-b84bd9e43b13 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:31.375716 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e7b0039a-ac8b-48d0-ba0c-a9b70e02e95e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5375dcbca25d4515b2711c30ca1a54f3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:31.381158 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0076c-8271-46e6-a3b4-e6a68aee57c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:31.381897 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0076c-8271-46e6-a3b4-e6a68aee57c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:31.382253 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0076c-8271-46e6-a3b4-e6a68aee57c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:31.382630 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0076c-8271-46e6-a3b4-e6a68aee57c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:31.383351 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0076c-8271-46e6-a3b4-e6a68aee57c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:31.384419 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e7b0039a-ac8b-48d0-ba0c-a9b70e02e95e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5375dcbca25d4515b2711c30ca1a54f3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:31.385144 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e7b0039a-ac8b-48d0-ba0c-a9b70e02e95e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5375dcbca25d4515b2711c30ca1a54f3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:31.385253 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0076c-8271-46e6-a3b4-e6a68aee57c9 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:31.385780 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e7b0039a-ac8b-48d0-ba0c-a9b70e02e95e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5375dcbca25d4515b2711c30ca1a54f3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:31.388439 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:31.403573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1094/2218] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:21:31.405483 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0076c-8271-46e6-a3b4-e6a68aee57c9 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:31.407430 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-1cd0076c-8271-46e6-a3b4-e6a68aee57c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:31.412374 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e7b0039a-ac8b-48d0-ba0c-a9b70e02e95e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5375dcbca25d4515b2711c30ca1a54f3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:31.422189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0076c-8271-46e6-a3b4-e6a68aee57c9 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:31.427718 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6680c13e-a984-44d4-b553-34867abf8d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:31.428338 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680c13e-a984-44d4-b553-34867abf8d91 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:31.428619 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680c13e-a984-44d4-b553-34867abf8d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:31.428893 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680c13e-a984-44d4-b553-34867abf8d91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:31.429551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6680c13e-a984-44d4-b553-34867abf8d91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:31.431238 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1125/2219] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:21:31 2022] DELETE /identity/v3/users/5375dcbca25d4515b2711c30ca1a54f3 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:31.431355 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6680c13e-a984-44d4-b553-34867abf8d91 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:31.443197 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1095/2220] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:21:31.449939 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0039a-ac8b-48d0-ba0c-a9b70e02e95e admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:31.531175 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0039a-ac8b-48d0-ba0c-a9b70e02e95e admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:31.536177 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f1625-4567-43c7-9369-53d7ed43d394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:31.536896 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f1625-4567-43c7-9369-53d7ed43d394 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:31.537184 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f1625-4567-43c7-9369-53d7ed43d394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:31.537511 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f1625-4567-43c7-9369-53d7ed43d394 admin admin] PATH_INFO: `/v3/projects/293e5e0fe0e4422a881f742c806bf167/users/00a46ad6e58243c5a112182d051523c3/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:31.555902 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f1625-4567-43c7-9369-53d7ed43d394 admin admin] RBAC: Authorizing `identity:create_grant(project_id=293e5e0fe0e4422a881f742c806bf167, user_id=00a46ad6e58243c5a112182d051523c3, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:31.561971 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f1625-4567-43c7-9369-53d7ed43d394 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:31.587732 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1126/2221] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:21:31 2022] PUT /identity/v3/projects/293e5e0fe0e4422a881f742c806bf167/users/00a46ad6e58243c5a112182d051523c3/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:31.592099 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0cce8-21c7-48de-89bd-ec6d21b52167 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:31.592386 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0cce8-21c7-48de-89bd-ec6d21b52167 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:31.592713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0cce8-21c7-48de-89bd-ec6d21b52167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:31.636482 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-82d0cce8-21c7-48de-89bd-ec6d21b52167 None None] MFA Rules not processed for user `00a46ad6e58243c5a112182d051523c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:31.637791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-82d0cce8-21c7-48de-89bd-ec6d21b52167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:31.691049 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:31.743269 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1096/2222] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:31.759307 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f1625-4567-43c7-9369-53d7ed43d394 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:31.763706 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-339f1625-4567-43c7-9369-53d7ed43d394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:31.829599 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0cce8-21c7-48de-89bd-ec6d21b52167 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:31.832070 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-82d0cce8-21c7-48de-89bd-ec6d21b52167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:31.863623 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f1625-4567-43c7-9369-53d7ed43d394 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:31.873362 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25905650-4e87-47fd-b28f-321411943cfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:31.874259 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25905650-4e87-47fd-b28f-321411943cfd service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:31.875942 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25905650-4e87-47fd-b28f-321411943cfd service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:31.876223 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25905650-4e87-47fd-b28f-321411943cfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:31.895187 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25905650-4e87-47fd-b28f-321411943cfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:31.896709 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25905650-4e87-47fd-b28f-321411943cfd service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:31.907241 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1127/2223] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:21:31.950681 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0cce8-21c7-48de-89bd-ec6d21b52167 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:31.956029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad679c7-b0b1-4465-995e-220ecc3ad23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:31.956769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad679c7-b0b1-4465-995e-220ecc3ad23f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:31.957116 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad679c7-b0b1-4465-995e-220ecc3ad23f admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:31.957452 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad679c7-b0b1-4465-995e-220ecc3ad23f admin admin] PATH_INFO: `/v3/projects/09d0fcf55384428d9d6b41047194a303` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:31.958996 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad679c7-b0b1-4465-995e-220ecc3ad23f admin admin] RBAC: Authorizing `identity:delete_project(project_id=09d0fcf55384428d9d6b41047194a303)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:31.961080 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad679c7-b0b1-4465-995e-220ecc3ad23f admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:32.023815 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3ad679c7-b0b1-4465-995e-220ecc3ad23f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09d0fcf55384428d9d6b41047194a303', 'request_id': 'req-3ad679c7-b0b1-4465-995e-220ecc3ad23f'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:32.034489 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3ad679c7-b0b1-4465-995e-220ecc3ad23f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09d0fcf55384428d9d6b41047194a303', 'request_id': 'req-3ad679c7-b0b1-4465-995e-220ecc3ad23f'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:32.042143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3ad679c7-b0b1-4465-995e-220ecc3ad23f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09d0fcf55384428d9d6b41047194a303', 'request_id': 'req-3ad679c7-b0b1-4465-995e-220ecc3ad23f'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:32.062851 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3ad679c7-b0b1-4465-995e-220ecc3ad23f admin admin] The token cache is being invalidate because project 09d0fcf55384428d9d6b41047194a303 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:21:32.063655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3ad679c7-b0b1-4465-995e-220ecc3ad23f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:32.064858 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3ad679c7-b0b1-4465-995e-220ecc3ad23f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:32.067366 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1097/2224] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:21:31 2022] DELETE /identity/v3/projects/09d0fcf55384428d9d6b41047194a303 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:32.077339 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee9223-a529-4db6-a9f3-b99bb3d9ae3f None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:32.077697 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee9223-a529-4db6-a9f3-b99bb3d9ae3f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:32.078029 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee9223-a529-4db6-a9f3-b99bb3d9ae3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:32.116313 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-76ee9223-a529-4db6-a9f3-b99bb3d9ae3f None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:32.117919 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-76ee9223-a529-4db6-a9f3-b99bb3d9ae3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:32.248074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1128/2225] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:32.258254 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6dcc8-21aa-41e3-8956-f3dcbdba8827 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:32.258768 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6dcc8-21aa-41e3-8956-f3dcbdba8827 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:32.258873 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6dcc8-21aa-41e3-8956-f3dcbdba8827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:32.317927 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-dde6dcc8-21aa-41e3-8956-f3dcbdba8827 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:32.321299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-dde6dcc8-21aa-41e3-8956-f3dcbdba8827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:32.347337 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1098/2226] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:32.349006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee9223-a529-4db6-a9f3-b99bb3d9ae3f None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:32.354276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:32.363036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee9223-a529-4db6-a9f3-b99bb3d9ae3f None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:32.384565 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe955e-f5f6-41da-bb9d-b11c728df626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:32.385238 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe955e-f5f6-41da-bb9d-b11c728df626 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:32.385669 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe955e-f5f6-41da-bb9d-b11c728df626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:32.386017 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe955e-f5f6-41da-bb9d-b11c728df626 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:32.386836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe955e-f5f6-41da-bb9d-b11c728df626 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:21:32.387219 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe955e-f5f6-41da-bb9d-b11c728df626 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:32.389047 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe955e-f5f6-41da-bb9d-b11c728df626 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:32.407581 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1129/2227] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:21:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:21:32.409717 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6dcc8-21aa-41e3-8956-f3dcbdba8827 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:32.421029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6dcc8-21aa-41e3-8956-f3dcbdba8827 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:32.427261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe4c11-9cf8-4eb1-b98d-d49415d31f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:32.428090 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe4c11-9cf8-4eb1-b98d-d49415d31f14 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:32.428438 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe4c11-9cf8-4eb1-b98d-d49415d31f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:32.428847 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe4c11-9cf8-4eb1-b98d-d49415d31f14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:32.429579 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe4c11-9cf8-4eb1-b98d-d49415d31f14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:32.431680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe4c11-9cf8-4eb1-b98d-d49415d31f14 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:32.464728 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1099/2228] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:21:32 2022] POST /identity/v3/projects => generated 363 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:21:32.468573 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe955e-f5f6-41da-bb9d-b11c728df626 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:32.481949 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe955e-f5f6-41da-bb9d-b11c728df626 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:32.543958 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aad94b-aa78-49bc-8b16-03c55a15852f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:32.544487 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aad94b-aa78-49bc-8b16-03c55a15852f admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:32.544764 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aad94b-aa78-49bc-8b16-03c55a15852f admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:32.545032 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aad94b-aa78-49bc-8b16-03c55a15852f admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:32.545690 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aad94b-aa78-49bc-8b16-03c55a15852f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:32.547627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aad94b-aa78-49bc-8b16-03c55a15852f admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:32.569281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1130/2229] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:21:32 2022] POST /identity/v3/users => generated 282 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:21:32.573446 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe4c11-9cf8-4eb1-b98d-d49415d31f14 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:32.584229 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe4c11-9cf8-4eb1-b98d-d49415d31f14 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:32.588810 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751644d2-ef84-4ef8-9f27-91f3114fe781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:32.589393 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751644d2-ef84-4ef8-9f27-91f3114fe781 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:32.589671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751644d2-ef84-4ef8-9f27-91f3114fe781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:32.589945 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751644d2-ef84-4ef8-9f27-91f3114fe781 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:32.590613 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751644d2-ef84-4ef8-9f27-91f3114fe781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:32.592232 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751644d2-ef84-4ef8-9f27-91f3114fe781 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:32.602623 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1100/2230] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:21:32.606461 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aad94b-aa78-49bc-8b16-03c55a15852f admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:32.616422 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aad94b-aa78-49bc-8b16-03c55a15852f admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:32.620945 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638128f7-5ce2-4e2a-8937-6bdcbf9f2c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:32.621527 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638128f7-5ce2-4e2a-8937-6bdcbf9f2c9e admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:32.621803 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638128f7-5ce2-4e2a-8937-6bdcbf9f2c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:32.622067 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638128f7-5ce2-4e2a-8937-6bdcbf9f2c9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:32.622698 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638128f7-5ce2-4e2a-8937-6bdcbf9f2c9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:32.624314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638128f7-5ce2-4e2a-8937-6bdcbf9f2c9e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:32.634632 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1131/2231] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:21:32.638737 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751644d2-ef84-4ef8-9f27-91f3114fe781 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:32.649826 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751644d2-ef84-4ef8-9f27-91f3114fe781 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:32.654624 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282826d-e171-4a69-a715-0761eb1df817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:32.655284 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282826d-e171-4a69-a715-0761eb1df817 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:32.655587 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282826d-e171-4a69-a715-0761eb1df817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:32.655862 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282826d-e171-4a69-a715-0761eb1df817 admin admin] PATH_INFO: `/v3/projects/b8a8919343314702bbbb30f67a06d4db/users/a6f376032e1a44c7865f564fabcc66da/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:32.670838 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282826d-e171-4a69-a715-0761eb1df817 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8a8919343314702bbbb30f67a06d4db, user_id=a6f376032e1a44c7865f564fabcc66da, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:32.673542 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282826d-e171-4a69-a715-0761eb1df817 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:32.682180 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:32.687453 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1101/2232] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:21:32 2022] PUT /identity/v3/projects/b8a8919343314702bbbb30f67a06d4db/users/a6f376032e1a44c7865f564fabcc66da/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:32.697635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977538c-054a-4ca1-a68e-a883a5d4256c None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:32.697924 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977538c-054a-4ca1-a68e-a883a5d4256c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:32.698232 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977538c-054a-4ca1-a68e-a883a5d4256c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:32.727041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-f977538c-054a-4ca1-a68e-a883a5d4256c None None] MFA Rules not processed for user `a6f376032e1a44c7865f564fabcc66da`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:32.728323 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f977538c-054a-4ca1-a68e-a883a5d4256c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:32.793124 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1132/2233] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:32.804124 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282826d-e171-4a69-a715-0761eb1df817 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:32.805827 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-a282826d-e171-4a69-a715-0761eb1df817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:32.863019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282826d-e171-4a69-a715-0761eb1df817 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:32.866662 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83936d8d-7a33-4a5a-945c-ebcc22b3b8cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:32.867260 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83936d8d-7a33-4a5a-945c-ebcc22b3b8cb service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:32.867555 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83936d8d-7a33-4a5a-945c-ebcc22b3b8cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:32.867833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83936d8d-7a33-4a5a-945c-ebcc22b3b8cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:32.876229 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83936d8d-7a33-4a5a-945c-ebcc22b3b8cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:32.877645 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83936d8d-7a33-4a5a-945c-ebcc22b3b8cb service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:32.886501 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1102/2234] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:21:38.305390 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977538c-054a-4ca1-a68e-a883a5d4256c None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:38.306852 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f977538c-054a-4ca1-a68e-a883a5d4256c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:38.358709 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83936d8d-7a33-4a5a-945c-ebcc22b3b8cb service cinder] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:38.360131 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-83936d8d-7a33-4a5a-945c-ebcc22b3b8cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:38.377086 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:38.383810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977538c-054a-4ca1-a68e-a883a5d4256c None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:38.394063 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53539cc-d545-4b18-bad3-cfed046a2756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:38.394836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53539cc-d545-4b18-bad3-cfed046a2756 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:38.395270 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53539cc-d545-4b18-bad3-cfed046a2756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:38.395803 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53539cc-d545-4b18-bad3-cfed046a2756 admin admin] PATH_INFO: `/v3/users/7e5b55cdebe34ea7ac7d26986e1e4ea2` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:38.397216 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53539cc-d545-4b18-bad3-cfed046a2756 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e5b55cdebe34ea7ac7d26986e1e4ea2)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:38.399370 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53539cc-d545-4b18-bad3-cfed046a2756 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:38.456751 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83936d8d-7a33-4a5a-945c-ebcc22b3b8cb service cinder] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:38.463327 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35496639-8660-43d1-a1ad-c6bac37f1ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:38.464475 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35496639-8660-43d1-a1ad-c6bac37f1ec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:38.464917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35496639-8660-43d1-a1ad-c6bac37f1ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:38.465366 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35496639-8660-43d1-a1ad-c6bac37f1ec7 admin admin] PATH_INFO: `/v3/users/00a46ad6e58243c5a112182d051523c3` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:38.466948 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35496639-8660-43d1-a1ad-c6bac37f1ec7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00a46ad6e58243c5a112182d051523c3)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:38.470389 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35496639-8660-43d1-a1ad-c6bac37f1ec7 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:38.476244 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d53539cc-d545-4b18-bad3-cfed046a2756 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e5b55cdebe34ea7ac7d26986e1e4ea2'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:38.487393 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d53539cc-d545-4b18-bad3-cfed046a2756 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e5b55cdebe34ea7ac7d26986e1e4ea2'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:38.488177 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d53539cc-d545-4b18-bad3-cfed046a2756 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e5b55cdebe34ea7ac7d26986e1e4ea2'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:38.489178 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d53539cc-d545-4b18-bad3-cfed046a2756 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e5b55cdebe34ea7ac7d26986e1e4ea2'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:38.506333 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d53539cc-d545-4b18-bad3-cfed046a2756 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e5b55cdebe34ea7ac7d26986e1e4ea2'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:38.522561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1133/2235] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:21:38 2022] DELETE /identity/v3/users/7e5b55cdebe34ea7ac7d26986e1e4ea2 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:38.527351 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e10a3-d327-4557-8d63-ca4b3681c8ed None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:38.527663 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e10a3-d327-4557-8d63-ca4b3681c8ed None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:38.527939 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e10a3-d327-4557-8d63-ca4b3681c8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:38.528329 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-35496639-8660-43d1-a1ad-c6bac37f1ec7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00a46ad6e58243c5a112182d051523c3'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:38.538252 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-35496639-8660-43d1-a1ad-c6bac37f1ec7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00a46ad6e58243c5a112182d051523c3'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:38.538953 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-35496639-8660-43d1-a1ad-c6bac37f1ec7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00a46ad6e58243c5a112182d051523c3'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:38.550304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-8c3e10a3-d327-4557-8d63-ca4b3681c8ed None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:38.551654 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8c3e10a3-d327-4557-8d63-ca4b3681c8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:38.552899 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-35496639-8660-43d1-a1ad-c6bac37f1ec7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00a46ad6e58243c5a112182d051523c3'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:38.560144 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-35496639-8660-43d1-a1ad-c6bac37f1ec7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00a46ad6e58243c5a112182d051523c3'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:38.566128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1103/2236] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:21:38 2022] DELETE /identity/v3/users/00a46ad6e58243c5a112182d051523c3 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:38.570472 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d982b-8cf6-4c85-addb-757186a917f9 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:38.570760 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d982b-8cf6-4c85-addb-757186a917f9 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:38.571035 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d982b-8cf6-4c85-addb-757186a917f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:38.600139 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-d89d982b-8cf6-4c85-addb-757186a917f9 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:38.601541 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-d89d982b-8cf6-4c85-addb-757186a917f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:38.625566 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1134/2237] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:38.632408 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:38.639726 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e10a3-d327-4557-8d63-ca4b3681c8ed None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:38.643139 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8c3e10a3-d327-4557-8d63-ca4b3681c8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:38.692871 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1104/2238] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:21:38.704700 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d982b-8cf6-4c85-addb-757186a917f9 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:38.706769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-d89d982b-8cf6-4c85-addb-757186a917f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:38.731719 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e10a3-d327-4557-8d63-ca4b3681c8ed None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:38.776128 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:38.803190 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d982b-8cf6-4c85-addb-757186a917f9 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:38.803461 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881abd45-4a84-4190-9509-2a82984c2b46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:38.803869 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881abd45-4a84-4190-9509-2a82984c2b46 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:38.804194 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881abd45-4a84-4190-9509-2a82984c2b46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:38.804542 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881abd45-4a84-4190-9509-2a82984c2b46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:38.807781 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78fa35a-735a-49d7-b896-7bf27a5ac463 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:38.808497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fa35a-735a-49d7-b896-7bf27a5ac463 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:38.808849 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fa35a-735a-49d7-b896-7bf27a5ac463 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:38.809130 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fa35a-735a-49d7-b896-7bf27a5ac463 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:38.816456 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881abd45-4a84-4190-9509-2a82984c2b46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:38.818248 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881abd45-4a84-4190-9509-2a82984c2b46 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:38.826511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78fa35a-735a-49d7-b896-7bf27a5ac463 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:38.827804 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78fa35a-735a-49d7-b896-7bf27a5ac463 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:38.832723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1135/2239] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:21:38.840062 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1105/2240] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:21:39.369611 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881abd45-4a84-4190-9509-2a82984c2b46 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:39.372133 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-881abd45-4a84-4190-9509-2a82984c2b46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:39.387148 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881abd45-4a84-4190-9509-2a82984c2b46 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:39.392020 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc4ce9-63ca-4914-ad0e-b979e06313da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:39.392714 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc4ce9-63ca-4914-ad0e-b979e06313da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:39.393000 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc4ce9-63ca-4914-ad0e-b979e06313da admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:39.393276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc4ce9-63ca-4914-ad0e-b979e06313da admin admin] PATH_INFO: `/v3/projects/629a461f93fc4e77ba73c47b66cc7814` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:39.398050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc4ce9-63ca-4914-ad0e-b979e06313da admin admin] RBAC: Authorizing `identity:delete_project(project_id=629a461f93fc4e77ba73c47b66cc7814)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:39.398050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc4ce9-63ca-4914-ad0e-b979e06313da admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:39.415349 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78fa35a-735a-49d7-b896-7bf27a5ac463 service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:39.417150 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-d78fa35a-735a-49d7-b896-7bf27a5ac463 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:39.432299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78fa35a-735a-49d7-b896-7bf27a5ac463 service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:39.438518 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdefbc9f-ccf8-4ded-b15d-b0a31cc8f8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:39.439100 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdefbc9f-ccf8-4ded-b15d-b0a31cc8f8dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:39.439492 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdefbc9f-ccf8-4ded-b15d-b0a31cc8f8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:39.440317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdefbc9f-ccf8-4ded-b15d-b0a31cc8f8dc admin admin] PATH_INFO: `/v3/projects/293e5e0fe0e4422a881f742c806bf167` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:39.441672 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdefbc9f-ccf8-4ded-b15d-b0a31cc8f8dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=293e5e0fe0e4422a881f742c806bf167)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:39.447713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdefbc9f-ccf8-4ded-b15d-b0a31cc8f8dc admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:39.472857 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a3cc4ce9-63ca-4914-ad0e-b979e06313da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '629a461f93fc4e77ba73c47b66cc7814', 'request_id': 'req-a3cc4ce9-63ca-4914-ad0e-b979e06313da'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:39.481283 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a3cc4ce9-63ca-4914-ad0e-b979e06313da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '629a461f93fc4e77ba73c47b66cc7814', 'request_id': 'req-a3cc4ce9-63ca-4914-ad0e-b979e06313da'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:39.491430 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a3cc4ce9-63ca-4914-ad0e-b979e06313da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '629a461f93fc4e77ba73c47b66cc7814', 'request_id': 'req-a3cc4ce9-63ca-4914-ad0e-b979e06313da'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:39.504942 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-fdefbc9f-ccf8-4ded-b15d-b0a31cc8f8dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '293e5e0fe0e4422a881f742c806bf167', 'request_id': 'req-fdefbc9f-ccf8-4ded-b15d-b0a31cc8f8dc'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:39.512024 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a3cc4ce9-63ca-4914-ad0e-b979e06313da admin admin] The token cache is being invalidate because project 629a461f93fc4e77ba73c47b66cc7814 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:21:39.512458 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a3cc4ce9-63ca-4914-ad0e-b979e06313da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:39.513336 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a3cc4ce9-63ca-4914-ad0e-b979e06313da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:39.514473 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-fdefbc9f-ccf8-4ded-b15d-b0a31cc8f8dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '293e5e0fe0e4422a881f742c806bf167', 'request_id': 'req-fdefbc9f-ccf8-4ded-b15d-b0a31cc8f8dc'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:39.515207 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1136/2241] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:21:39 2022] DELETE /identity/v3/projects/629a461f93fc4e77ba73c47b66cc7814 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:39.520359 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc4ce9-63ca-4914-ad0e-b979e06313da admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:39.522378 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a3cc4ce9-63ca-4914-ad0e-b979e06313da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:39.522927 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-fdefbc9f-ccf8-4ded-b15d-b0a31cc8f8dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '293e5e0fe0e4422a881f742c806bf167', 'request_id': 'req-fdefbc9f-ccf8-4ded-b15d-b0a31cc8f8dc'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:39.544643 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-fdefbc9f-ccf8-4ded-b15d-b0a31cc8f8dc admin admin] The token cache is being invalidate because project 293e5e0fe0e4422a881f742c806bf167 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:21:39.544643 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-fdefbc9f-ccf8-4ded-b15d-b0a31cc8f8dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:39.545989 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-fdefbc9f-ccf8-4ded-b15d-b0a31cc8f8dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:39.547911 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1106/2242] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:21:39 2022] DELETE /identity/v3/projects/293e5e0fe0e4422a881f742c806bf167 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:39.553694 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71145b4e-b880-47b4-a959-f0f00c54a46d None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:39.553961 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71145b4e-b880-47b4-a959-f0f00c54a46d None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:39.554242 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71145b4e-b880-47b4-a959-f0f00c54a46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:39.576698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-71145b4e-b880-47b4-a959-f0f00c54a46d None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:39.577946 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-71145b4e-b880-47b4-a959-f0f00c54a46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:39.600817 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc4ce9-63ca-4914-ad0e-b979e06313da admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:39.630832 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:39.660293 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1107/2243] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:21:39.669426 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e74ad-d170-4c7c-928c-b4996e9b69fe None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:39.669715 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e74ad-d170-4c7c-928c-b4996e9b69fe None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:39.669991 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e74ad-d170-4c7c-928c-b4996e9b69fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:39.682984 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09c697-583f-4d5e-ba7e-2f44ec8e3f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:39.684109 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09c697-583f-4d5e-ba7e-2f44ec8e3f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:39.684632 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09c697-583f-4d5e-ba7e-2f44ec8e3f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:39.685091 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09c697-583f-4d5e-ba7e-2f44ec8e3f60 admin admin] PATH_INFO: `/v3/users/241425bea3504f98af1f8a6d23617b7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:39.686519 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c09c697-583f-4d5e-ba7e-2f44ec8e3f60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=241425bea3504f98af1f8a6d23617b7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:39.688697 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c09c697-583f-4d5e-ba7e-2f44ec8e3f60 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:39.693063 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-6d3e74ad-d170-4c7c-928c-b4996e9b69fe None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:39.694471 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-6d3e74ad-d170-4c7c-928c-b4996e9b69fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:39.697057 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:39.706516 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1108/2244] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:21:39.712924 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e74ad-d170-4c7c-928c-b4996e9b69fe None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:39.724284 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e74ad-d170-4c7c-928c-b4996e9b69fe None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:39.729776 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbd388-d907-4eda-bb56-8156a84a8344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:39.730597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbd388-d907-4eda-bb56-8156a84a8344 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:39.730880 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbd388-d907-4eda-bb56-8156a84a8344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:39.731156 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbd388-d907-4eda-bb56-8156a84a8344 admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:39.731979 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dbd388-d907-4eda-bb56-8156a84a8344 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:21:39.732294 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dbd388-d907-4eda-bb56-8156a84a8344 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:39.734289 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dbd388-d907-4eda-bb56-8156a84a8344 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:39.750750 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8c09c697-583f-4d5e-ba7e-2f44ec8e3f60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '241425bea3504f98af1f8a6d23617b7f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:39.755120 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1109/2245] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:21:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:21:39.759719 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbd388-d907-4eda-bb56-8156a84a8344 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:39.762383 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8c09c697-583f-4d5e-ba7e-2f44ec8e3f60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '241425bea3504f98af1f8a6d23617b7f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:39.763109 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8c09c697-583f-4d5e-ba7e-2f44ec8e3f60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '241425bea3504f98af1f8a6d23617b7f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:39.763772 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8c09c697-583f-4d5e-ba7e-2f44ec8e3f60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '241425bea3504f98af1f8a6d23617b7f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:39.771104 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbd388-d907-4eda-bb56-8156a84a8344 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:39.777140 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe5120-5a73-47c5-a526-d31d55a82de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:39.777711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8c09c697-583f-4d5e-ba7e-2f44ec8e3f60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '241425bea3504f98af1f8a6d23617b7f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:39.777830 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe5120-5a73-47c5-a526-d31d55a82de2 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:39.778002 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe5120-5a73-47c5-a526-d31d55a82de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:39.778267 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe5120-5a73-47c5-a526-d31d55a82de2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:39.778902 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe5120-5a73-47c5-a526-d31d55a82de2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:39.780863 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe5120-5a73-47c5-a526-d31d55a82de2 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:39.798275 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1137/2246] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:21:39 2022] DELETE /identity/v3/users/241425bea3504f98af1f8a6d23617b7f => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:39.823601 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1110/2247] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:21:39 2022] POST /identity/v3/projects => generated 377 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:21:39.826506 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09c697-583f-4d5e-ba7e-2f44ec8e3f60 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:39.828191 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8c09c697-583f-4d5e-ba7e-2f44ec8e3f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:39.921533 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09c697-583f-4d5e-ba7e-2f44ec8e3f60 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:40.031847 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce786220-425a-4929-b8b1-adc1b11bbdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:40.032834 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce786220-425a-4929-b8b1-adc1b11bbdde admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:40.033229 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce786220-425a-4929-b8b1-adc1b11bbdde admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:40.033931 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce786220-425a-4929-b8b1-adc1b11bbdde admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:40.036529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce786220-425a-4929-b8b1-adc1b11bbdde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:40.038725 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce786220-425a-4929-b8b1-adc1b11bbdde admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:40.069170 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1138/2248] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:21:39 2022] POST /identity/v3/users => generated 289 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:21:40.070357 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe5120-5a73-47c5-a526-d31d55a82de2 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:40.082586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe5120-5a73-47c5-a526-d31d55a82de2 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:40.091581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6ea8e-7d24-494d-880b-7958bd89096e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:40.092952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6ea8e-7d24-494d-880b-7958bd89096e admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:40.092952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6ea8e-7d24-494d-880b-7958bd89096e admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:40.093425 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6ea8e-7d24-494d-880b-7958bd89096e admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:40.094196 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6ea8e-7d24-494d-880b-7958bd89096e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:40.096435 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6ea8e-7d24-494d-880b-7958bd89096e admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:40.108365 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce786220-425a-4929-b8b1-adc1b11bbdde admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:40.110636 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1111/2249] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:21:40.111182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ce786220-425a-4929-b8b1-adc1b11bbdde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:40.115786 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6ea8e-7d24-494d-880b-7958bd89096e admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:40.124946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce786220-425a-4929-b8b1-adc1b11bbdde admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:40.128211 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6ea8e-7d24-494d-880b-7958bd89096e admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:40.133170 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175a95a-6a8e-4fbc-8ab1-601d2383ec2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:40.133853 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175a95a-6a8e-4fbc-8ab1-601d2383ec2b admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:40.134207 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175a95a-6a8e-4fbc-8ab1-601d2383ec2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:40.134550 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e77b1e-f933-41c3-a5a7-4be78fe34be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:40.134670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175a95a-6a8e-4fbc-8ab1-601d2383ec2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:40.135580 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b175a95a-6a8e-4fbc-8ab1-601d2383ec2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:40.135701 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e77b1e-f933-41c3-a5a7-4be78fe34be9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:40.136111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e77b1e-f933-41c3-a5a7-4be78fe34be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:40.136394 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e77b1e-f933-41c3-a5a7-4be78fe34be9 admin admin] PATH_INFO: `/v3/users/64b1a765aaf04a649dd37d748c2c57f1` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:40.137586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b175a95a-6a8e-4fbc-8ab1-601d2383ec2b admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:40.138366 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e77b1e-f933-41c3-a5a7-4be78fe34be9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64b1a765aaf04a649dd37d748c2c57f1)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:40.140993 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e77b1e-f933-41c3-a5a7-4be78fe34be9 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:40.154142 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1112/2250] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:21:40.155018 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175a95a-6a8e-4fbc-8ab1-601d2383ec2b admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:40.164050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:40.165183 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-b175a95a-6a8e-4fbc-8ab1-601d2383ec2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:40.182131 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175a95a-6a8e-4fbc-8ab1-601d2383ec2b admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:40.187338 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42376f5-6fb6-42b8-8fbb-2f530fb8f9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:40.188161 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42376f5-6fb6-42b8-8fbb-2f530fb8f9cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:40.188563 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42376f5-6fb6-42b8-8fbb-2f530fb8f9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:40.188832 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42376f5-6fb6-42b8-8fbb-2f530fb8f9cf admin admin] PATH_INFO: `/v3/projects/8a64774d68864e4fbf1e19f32be643bb` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:40.190552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42376f5-6fb6-42b8-8fbb-2f530fb8f9cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a64774d68864e4fbf1e19f32be643bb)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:40.192555 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42376f5-6fb6-42b8-8fbb-2f530fb8f9cf admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:40.196565 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e2e77b1e-f933-41c3-a5a7-4be78fe34be9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64b1a765aaf04a649dd37d748c2c57f1'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:40.203854 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e2e77b1e-f933-41c3-a5a7-4be78fe34be9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64b1a765aaf04a649dd37d748c2c57f1'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:40.204734 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e2e77b1e-f933-41c3-a5a7-4be78fe34be9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64b1a765aaf04a649dd37d748c2c57f1'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:40.205378 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e2e77b1e-f933-41c3-a5a7-4be78fe34be9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64b1a765aaf04a649dd37d748c2c57f1'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:40.218293 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e2e77b1e-f933-41c3-a5a7-4be78fe34be9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64b1a765aaf04a649dd37d748c2c57f1'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:40.232398 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1139/2251] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:21:40 2022] DELETE /identity/v3/users/64b1a765aaf04a649dd37d748c2c57f1 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:21:40.233417 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e77b1e-f933-41c3-a5a7-4be78fe34be9 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:40.237096 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e2e77b1e-f933-41c3-a5a7-4be78fe34be9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:40.249724 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-d42376f5-6fb6-42b8-8fbb-2f530fb8f9cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a64774d68864e4fbf1e19f32be643bb', 'request_id': 'req-d42376f5-6fb6-42b8-8fbb-2f530fb8f9cf'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:40.258009 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-d42376f5-6fb6-42b8-8fbb-2f530fb8f9cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a64774d68864e4fbf1e19f32be643bb', 'request_id': 'req-d42376f5-6fb6-42b8-8fbb-2f530fb8f9cf'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:40.264129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-d42376f5-6fb6-42b8-8fbb-2f530fb8f9cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a64774d68864e4fbf1e19f32be643bb', 'request_id': 'req-d42376f5-6fb6-42b8-8fbb-2f530fb8f9cf'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:40.276207 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-d42376f5-6fb6-42b8-8fbb-2f530fb8f9cf admin admin] The token cache is being invalidate because project 8a64774d68864e4fbf1e19f32be643bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:21:40.276420 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-d42376f5-6fb6-42b8-8fbb-2f530fb8f9cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:40.277088 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-d42376f5-6fb6-42b8-8fbb-2f530fb8f9cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:40.278608 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1113/2252] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:21:40 2022] DELETE /identity/v3/projects/8a64774d68864e4fbf1e19f32be643bb => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:40.279922 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab02038-0299-473e-b113-87c113ad58a7 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:40.280201 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab02038-0299-473e-b113-87c113ad58a7 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:40.280477 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab02038-0299-473e-b113-87c113ad58a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:40.300570 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-fab02038-0299-473e-b113-87c113ad58a7 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:40.301944 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-fab02038-0299-473e-b113-87c113ad58a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:40.306730 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e77b1e-f933-41c3-a5a7-4be78fe34be9 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:40.309340 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1114/2253] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:40.310151 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab02038-0299-473e-b113-87c113ad58a7 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:40.311353 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-fab02038-0299-473e-b113-87c113ad58a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:40.314356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff9556-ed36-48cb-9ecc-25c11b53ac6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:40.315704 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff9556-ed36-48cb-9ecc-25c11b53ac6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:40.316091 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff9556-ed36-48cb-9ecc-25c11b53ac6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:40.316473 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff9556-ed36-48cb-9ecc-25c11b53ac6b admin admin] PATH_INFO: `/v3/projects/e20da78cb53f46a7876263720edb8830/users/dba21c47c98441dd857160264520dc9d/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:40.323692 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab02038-0299-473e-b113-87c113ad58a7 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:40.328770 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5272e0-c076-4503-94f9-c2839920593f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:40.329382 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5272e0-c076-4503-94f9-c2839920593f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:40.329657 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5272e0-c076-4503-94f9-c2839920593f admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:40.329926 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5272e0-c076-4503-94f9-c2839920593f admin admin] PATH_INFO: `/v3/users/217bce795ae04ee280071224db6d26d4` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:40.331090 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5272e0-c076-4503-94f9-c2839920593f admin admin] RBAC: Authorizing `identity:delete_user(user_id=217bce795ae04ee280071224db6d26d4)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:40.332776 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff9556-ed36-48cb-9ecc-25c11b53ac6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e20da78cb53f46a7876263720edb8830, user_id=dba21c47c98441dd857160264520dc9d, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:40.333025 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5272e0-c076-4503-94f9-c2839920593f admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:40.336496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff9556-ed36-48cb-9ecc-25c11b53ac6b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:40.352738 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1140/2254] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:21:40 2022] PUT /identity/v3/projects/e20da78cb53f46a7876263720edb8830/users/dba21c47c98441dd857160264520dc9d/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:40.353814 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff9556-ed36-48cb-9ecc-25c11b53ac6b admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:40.366223 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c5ff9556-ed36-48cb-9ecc-25c11b53ac6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:40.404920 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-6f5272e0-c076-4503-94f9-c2839920593f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '217bce795ae04ee280071224db6d26d4'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:40.416100 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-6f5272e0-c076-4503-94f9-c2839920593f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '217bce795ae04ee280071224db6d26d4'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:40.416742 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-6f5272e0-c076-4503-94f9-c2839920593f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '217bce795ae04ee280071224db6d26d4'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:40.419027 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=82,keystone:INSERT=3,keystone:DELETE=15 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:40.436845 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-6f5272e0-c076-4503-94f9-c2839920593f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '217bce795ae04ee280071224db6d26d4'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:40.447008 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff9556-ed36-48cb-9ecc-25c11b53ac6b admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:40.449589 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-6f5272e0-c076-4503-94f9-c2839920593f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '217bce795ae04ee280071224db6d26d4'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:40.451985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1caec-b289-499c-bfde-0254bb7339d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:40.452601 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1caec-b289-499c-bfde-0254bb7339d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:40.452876 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1caec-b289-499c-bfde-0254bb7339d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:40.453148 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1caec-b289-499c-bfde-0254bb7339d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:40.454948 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-87f1caec-b289-499c-bfde-0254bb7339d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:40.456584 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1115/2255] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:21:40 2022] DELETE /identity/v3/users/217bce795ae04ee280071224db6d26d4 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:40.458147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8ce87-e89c-40fa-8aec-c0360d55c576 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:40.458525 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8ce87-e89c-40fa-8aec-c0360d55c576 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:40.458899 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8ce87-e89c-40fa-8aec-c0360d55c576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:40.508024 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:40.514707 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-36f8ce87-e89c-40fa-8aec-c0360d55c576 None None] MFA Rules not processed for user `dba21c47c98441dd857160264520dc9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:40.516650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-36f8ce87-e89c-40fa-8aec-c0360d55c576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:40.563766 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1caec-b289-499c-bfde-0254bb7339d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:40.565309 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1caec-b289-499c-bfde-0254bb7339d5 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:40.576599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1141/2256] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:21:40.604232 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1116/2257] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:40.608981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1caec-b289-499c-bfde-0254bb7339d5 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:40.610425 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-87f1caec-b289-499c-bfde-0254bb7339d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:40.623244 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1caec-b289-499c-bfde-0254bb7339d5 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:40.733189 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7fefbe-c4db-4648-9f1a-c62fbeaece22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:40.734102 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7fefbe-c4db-4648-9f1a-c62fbeaece22 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:40.734397 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7fefbe-c4db-4648-9f1a-c62fbeaece22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:40.734685 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7fefbe-c4db-4648-9f1a-c62fbeaece22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:40.735435 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7fefbe-c4db-4648-9f1a-c62fbeaece22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:40.740180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7fefbe-c4db-4648-9f1a-c62fbeaece22 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:40.780545 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1142/2258] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:21:40 2022] POST /identity/v3/projects => generated 375 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:21:40.786873 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8ce87-e89c-40fa-8aec-c0360d55c576 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:40.811231 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8ce87-e89c-40fa-8aec-c0360d55c576 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:40.907735 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7fefbe-c4db-4648-9f1a-c62fbeaece22 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:40.910563 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2b7fefbe-c4db-4648-9f1a-c62fbeaece22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:40.920134 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec01353-3744-48ac-8cd7-989b75a80e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:40.920819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec01353-3744-48ac-8cd7-989b75a80e11 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:40.921099 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec01353-3744-48ac-8cd7-989b75a80e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:40.921367 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec01353-3744-48ac-8cd7-989b75a80e11 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:40.922197 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec01353-3744-48ac-8cd7-989b75a80e11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:40.924311 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec01353-3744-48ac-8cd7-989b75a80e11 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:40.949839 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1117/2259] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:21:40 2022] POST /identity/v3/users => generated 288 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:21:40.954018 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec01353-3744-48ac-8cd7-989b75a80e11 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:40.965946 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec01353-3744-48ac-8cd7-989b75a80e11 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:40.972434 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d7b28-d38e-494c-94ee-5570aee89087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:40.973309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d7b28-d38e-494c-94ee-5570aee89087 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:40.973760 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d7b28-d38e-494c-94ee-5570aee89087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:40.974188 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d7b28-d38e-494c-94ee-5570aee89087 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:40.974995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096d7b28-d38e-494c-94ee-5570aee89087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:40.977238 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096d7b28-d38e-494c-94ee-5570aee89087 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:40.991646 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1118/2260] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:21:40.993323 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d7b28-d38e-494c-94ee-5570aee89087 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:41.001793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7fefbe-c4db-4648-9f1a-c62fbeaece22 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:41.005942 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d7b28-d38e-494c-94ee-5570aee89087 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:41.011119 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691cb8bc-e47c-4cb3-b333-ced9f23e389e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:41.011777 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cb8bc-e47c-4cb3-b333-ced9f23e389e admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:41.014250 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cb8bc-e47c-4cb3-b333-ced9f23e389e admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:41.014250 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cb8bc-e47c-4cb3-b333-ced9f23e389e admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:41.014250 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691cb8bc-e47c-4cb3-b333-ced9f23e389e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:41.014570 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691cb8bc-e47c-4cb3-b333-ced9f23e389e admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:41.021935 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc1ee07-062b-4714-b36a-c74560b3b587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:41.022647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1ee07-062b-4714-b36a-c74560b3b587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:41.026954 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1119/2261] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:21:41.030328 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1ee07-062b-4714-b36a-c74560b3b587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:41.030328 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1ee07-062b-4714-b36a-c74560b3b587 admin admin] PATH_INFO: `/v3/projects/606b96a009524ac0908d37fce333a84f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:41.030981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc1ee07-062b-4714-b36a-c74560b3b587 admin admin] RBAC: Authorizing `identity:delete_project(project_id=606b96a009524ac0908d37fce333a84f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:41.031097 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691cb8bc-e47c-4cb3-b333-ced9f23e389e admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:41.033001 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc1ee07-062b-4714-b36a-c74560b3b587 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:41.042879 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691cb8bc-e47c-4cb3-b333-ced9f23e389e admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:41.051573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7882a9ad-01ef-4a2a-a416-3f8cbe165be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:41.052282 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882a9ad-01ef-4a2a-a416-3f8cbe165be3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:41.052559 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882a9ad-01ef-4a2a-a416-3f8cbe165be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:41.052835 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882a9ad-01ef-4a2a-a416-3f8cbe165be3 admin admin] PATH_INFO: `/v3/projects/b97a84390f4e437d949342fe466ae89a/users/8ecc620b7e3f4eafbc304ecc0c64ccc0/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:41.071426 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7882a9ad-01ef-4a2a-a416-3f8cbe165be3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b97a84390f4e437d949342fe466ae89a, user_id=8ecc620b7e3f4eafbc304ecc0c64ccc0, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:41.074861 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7882a9ad-01ef-4a2a-a416-3f8cbe165be3 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:41.095617 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1120/2262] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:21:41 2022] PUT /identity/v3/projects/b97a84390f4e437d949342fe466ae89a/users/8ecc620b7e3f4eafbc304ecc0c64ccc0/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:21:41.098242 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7882a9ad-01ef-4a2a-a416-3f8cbe165be3 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:41.100457 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-7882a9ad-01ef-4a2a-a416-3f8cbe165be3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:41.112664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:41.137410 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ebc1ee07-062b-4714-b36a-c74560b3b587 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '606b96a009524ac0908d37fce333a84f', 'request_id': 'req-ebc1ee07-062b-4714-b36a-c74560b3b587'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:41.143924 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:41.148307 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ebc1ee07-062b-4714-b36a-c74560b3b587 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '606b96a009524ac0908d37fce333a84f', 'request_id': 'req-ebc1ee07-062b-4714-b36a-c74560b3b587'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:41.160345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ebc1ee07-062b-4714-b36a-c74560b3b587 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '606b96a009524ac0908d37fce333a84f', 'request_id': 'req-ebc1ee07-062b-4714-b36a-c74560b3b587'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:41.183904 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ebc1ee07-062b-4714-b36a-c74560b3b587 admin admin] The token cache is being invalidate because project 606b96a009524ac0908d37fce333a84f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:21:41.184454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ebc1ee07-062b-4714-b36a-c74560b3b587 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:41.185337 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ebc1ee07-062b-4714-b36a-c74560b3b587 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:41.187137 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1143/2263] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:21:40 2022] DELETE /identity/v3/projects/606b96a009524ac0908d37fce333a84f => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:41.189057 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72982b29-0ab7-476f-ab00-9d0930019f77 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:41.190044 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72982b29-0ab7-476f-ab00-9d0930019f77 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:41.190447 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72982b29-0ab7-476f-ab00-9d0930019f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:41.200094 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7882a9ad-01ef-4a2a-a416-3f8cbe165be3 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:41.227934 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-72982b29-0ab7-476f-ab00-9d0930019f77 None None] MFA Rules not processed for user `8ecc620b7e3f4eafbc304ecc0c64ccc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:41.229454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-72982b29-0ab7-476f-ab00-9d0930019f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:41.285179 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea4934-d042-4e71-a52c-597bfb3cb054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:41.285834 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea4934-d042-4e71-a52c-597bfb3cb054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:41.286105 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea4934-d042-4e71-a52c-597bfb3cb054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:41.286381 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea4934-d042-4e71-a52c-597bfb3cb054 admin admin] PATH_INFO: `/v3/projects/8689e09a1efd49a6bad89984909b185a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:41.287582 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea4934-d042-4e71-a52c-597bfb3cb054 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8689e09a1efd49a6bad89984909b185a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:41.289552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea4934-d042-4e71-a52c-597bfb3cb054 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:41.324776 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1144/2264] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:41.326207 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3f75c-a001-4dad-b4f5-8cb15f8abefc None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:41.326566 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3f75c-a001-4dad-b4f5-8cb15f8abefc None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:41.326905 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3f75c-a001-4dad-b4f5-8cb15f8abefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:41.341784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-c3ea4934-d042-4e71-a52c-597bfb3cb054 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8689e09a1efd49a6bad89984909b185a', 'request_id': 'req-c3ea4934-d042-4e71-a52c-597bfb3cb054'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:41.348146 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-7aa3f75c-a001-4dad-b4f5-8cb15f8abefc None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:41.349442 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-c3ea4934-d042-4e71-a52c-597bfb3cb054 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8689e09a1efd49a6bad89984909b185a', 'request_id': 'req-c3ea4934-d042-4e71-a52c-597bfb3cb054'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:41.349577 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7aa3f75c-a001-4dad-b4f5-8cb15f8abefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:41.356269 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-c3ea4934-d042-4e71-a52c-597bfb3cb054 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8689e09a1efd49a6bad89984909b185a', 'request_id': 'req-c3ea4934-d042-4e71-a52c-597bfb3cb054'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:41.371655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-c3ea4934-d042-4e71-a52c-597bfb3cb054 admin admin] The token cache is being invalidate because project 8689e09a1efd49a6bad89984909b185a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:21:41.371655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-c3ea4934-d042-4e71-a52c-597bfb3cb054 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:41.372056 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-c3ea4934-d042-4e71-a52c-597bfb3cb054 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:41.373540 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1121/2265] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:21:41 2022] DELETE /identity/v3/projects/8689e09a1efd49a6bad89984909b185a => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:41.374887 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea4934-d042-4e71-a52c-597bfb3cb054 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:41.376244 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-c3ea4934-d042-4e71-a52c-597bfb3cb054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:41.411330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1145/2266] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:41.412620 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24119281-fae6-40bf-b3db-7c7ac2eb258f None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:41.412909 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24119281-fae6-40bf-b3db-7c7ac2eb258f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:41.413181 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24119281-fae6-40bf-b3db-7c7ac2eb258f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:41.432152 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:41.435238 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-24119281-fae6-40bf-b3db-7c7ac2eb258f None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:41.436492 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-24119281-fae6-40bf-b3db-7c7ac2eb258f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:41.445942 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1146/2267] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:41.453136 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9227f03c-8fb0-4442-9acf-b44b00b9b1a8 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:41.453737 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9227f03c-8fb0-4442-9acf-b44b00b9b1a8 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:41.453737 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9227f03c-8fb0-4442-9acf-b44b00b9b1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:41.457157 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea4934-d042-4e71-a52c-597bfb3cb054 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:41.462537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd459d-ea37-4efa-9cdd-14f2945fb0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:41.463378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd459d-ea37-4efa-9cdd-14f2945fb0d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:41.463679 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd459d-ea37-4efa-9cdd-14f2945fb0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:41.463952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd459d-ea37-4efa-9cdd-14f2945fb0d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:41.464602 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd459d-ea37-4efa-9cdd-14f2945fb0d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:41.466574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd459d-ea37-4efa-9cdd-14f2945fb0d3 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:41.475405 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-9227f03c-8fb0-4442-9acf-b44b00b9b1a8 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:41.476810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9227f03c-8fb0-4442-9acf-b44b00b9b1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:41.487647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1147/2268] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:41.489240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05778a08-cba5-4538-8f27-fec4b81a667a None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:41.489533 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05778a08-cba5-4538-8f27-fec4b81a667a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:41.489828 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05778a08-cba5-4538-8f27-fec4b81a667a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:41.499200 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1122/2269] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:21:41 2022] POST /identity/v3/projects => generated 377 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:21:41.500287 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd459d-ea37-4efa-9cdd-14f2945fb0d3 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:41.510272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-05778a08-cba5-4538-8f27-fec4b81a667a None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:41.511636 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-05778a08-cba5-4538-8f27-fec4b81a667a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:41.577783 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:41.586696 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd459d-ea37-4efa-9cdd-14f2945fb0d3 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:41.591673 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c1881-88c7-4376-8a84-cdf978634655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:41.596765 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c1881-88c7-4376-8a84-cdf978634655 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:41.596941 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1148/2270] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:41.597053 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c1881-88c7-4376-8a84-cdf978634655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:41.597053 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c1881-88c7-4376-8a84-cdf978634655 admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:41.597220 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05778a08-cba5-4538-8f27-fec4b81a667a None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:41.597307 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4c1881-88c7-4376-8a84-cdf978634655 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:21:41.597307 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4c1881-88c7-4376-8a84-cdf978634655 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:41.597307 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4c1881-88c7-4376-8a84-cdf978634655 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:41.603494 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05778a08-cba5-4538-8f27-fec4b81a667a None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:41.610723 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1123/2271] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:21:41.612107 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7290c7c9-c7b1-44f4-b6fd-b7ceb23f50ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:41.612644 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c1881-88c7-4376-8a84-cdf978634655 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:41.614231 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290c7c9-c7b1-44f4-b6fd-b7ceb23f50ba admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:41.614532 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290c7c9-c7b1-44f4-b6fd-b7ceb23f50ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:41.614806 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290c7c9-c7b1-44f4-b6fd-b7ceb23f50ba admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:41.615454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7290c7c9-c7b1-44f4-b6fd-b7ceb23f50ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:41.617593 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7290c7c9-c7b1-44f4-b6fd-b7ceb23f50ba admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:41.623481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c1881-88c7-4376-8a84-cdf978634655 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:41.629960 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e535f7-ea21-4726-bdc7-a9cd6099fe1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:41.631025 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e535f7-ea21-4726-bdc7-a9cd6099fe1c admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:41.631453 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e535f7-ea21-4726-bdc7-a9cd6099fe1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:41.631722 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e535f7-ea21-4726-bdc7-a9cd6099fe1c admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:41.632711 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e535f7-ea21-4726-bdc7-a9cd6099fe1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:21:41.633182 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e535f7-ea21-4726-bdc7-a9cd6099fe1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:41.635350 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e535f7-ea21-4726-bdc7-a9cd6099fe1c admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:41.644131 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1149/2272] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:21:41 2022] POST /identity/v3/users => generated 295 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:21:41.645219 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7290c7c9-c7b1-44f4-b6fd-b7ceb23f50ba admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:41.651804 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1124/2273] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:21:41.652884 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e535f7-ea21-4726-bdc7-a9cd6099fe1c admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:41.660101 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7290c7c9-c7b1-44f4-b6fd-b7ceb23f50ba admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:41.662830 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e535f7-ea21-4726-bdc7-a9cd6099fe1c admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:41.665455 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa028a19-b1f8-4c03-9c31-cbc8929d833e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:41.666069 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa028a19-b1f8-4c03-9c31-cbc8929d833e admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:41.666522 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa028a19-b1f8-4c03-9c31-cbc8929d833e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:41.666807 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa028a19-b1f8-4c03-9c31-cbc8929d833e admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:41.667492 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa028a19-b1f8-4c03-9c31-cbc8929d833e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:41.668025 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9afe5c-79c1-40c5-b8d0-db3098a17ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:41.668613 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9afe5c-79c1-40c5-b8d0-db3098a17ae6 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:41.668897 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9afe5c-79c1-40c5-b8d0-db3098a17ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:41.669167 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9afe5c-79c1-40c5-b8d0-db3098a17ae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:41.669698 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa028a19-b1f8-4c03-9c31-cbc8929d833e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:41.669944 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9afe5c-79c1-40c5-b8d0-db3098a17ae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:41.671654 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9afe5c-79c1-40c5-b8d0-db3098a17ae6 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:41.685446 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1125/2274] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:21:41.686146 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9afe5c-79c1-40c5-b8d0-db3098a17ae6 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:41.698116 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9afe5c-79c1-40c5-b8d0-db3098a17ae6 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:41.703748 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f8e20-af8e-4f83-b330-83a43cc117da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:41.704405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f8e20-af8e-4f83-b330-83a43cc117da admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:41.704690 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f8e20-af8e-4f83-b330-83a43cc117da admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:41.704965 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f8e20-af8e-4f83-b330-83a43cc117da admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:41.705637 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137f8e20-af8e-4f83-b330-83a43cc117da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:41.707960 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137f8e20-af8e-4f83-b330-83a43cc117da admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:41.709471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1150/2275] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:21:41 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:21:41.713448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa028a19-b1f8-4c03-9c31-cbc8929d833e admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:41.742937 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1126/2276] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:21:41 2022] POST /identity/v3/projects => generated 399 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:21:41.743943 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f8e20-af8e-4f83-b330-83a43cc117da admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:41.793058 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa028a19-b1f8-4c03-9c31-cbc8929d833e admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:41.871124 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f8e20-af8e-4f83-b330-83a43cc117da admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:41.881314 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b9263-093a-431a-a72d-4cee0ad63da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:41.882681 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b9263-093a-431a-a72d-4cee0ad63da4 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:41.885037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b9263-093a-431a-a72d-4cee0ad63da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:41.886026 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b9263-093a-431a-a72d-4cee0ad63da4 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:41.887869 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425b9263-093a-431a-a72d-4cee0ad63da4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:41.892157 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425b9263-093a-431a-a72d-4cee0ad63da4 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:41.926129 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba2501d-0d7d-4289-9fa5-d9b4d9825b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:41.927735 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2501d-0d7d-4289-9fa5-d9b4d9825b6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:41.928390 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2501d-0d7d-4289-9fa5-d9b4d9825b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:41.929105 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2501d-0d7d-4289-9fa5-d9b4d9825b6f admin admin] PATH_INFO: `/v3/projects/1885171db7894a6497ded7c75e63ae52/users/42c3e53f1af4451498a55a591762517b/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:41.930262 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1127/2277] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:21:41 2022] POST /identity/v3/users => generated 286 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:21:41.931166 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b9263-093a-431a-a72d-4cee0ad63da4 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:41.950265 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b9263-093a-431a-a72d-4cee0ad63da4 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:41.956695 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc11eb-23d5-4c30-8cc4-6f0a5655092b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:41.957682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba2501d-0d7d-4289-9fa5-d9b4d9825b6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1885171db7894a6497ded7c75e63ae52, user_id=42c3e53f1af4451498a55a591762517b, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:41.957834 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc11eb-23d5-4c30-8cc4-6f0a5655092b admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:41.958075 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc11eb-23d5-4c30-8cc4-6f0a5655092b admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:41.958657 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc11eb-23d5-4c30-8cc4-6f0a5655092b admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:41.959947 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc11eb-23d5-4c30-8cc4-6f0a5655092b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:41.962823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc11eb-23d5-4c30-8cc4-6f0a5655092b admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:41.964138 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba2501d-0d7d-4289-9fa5-d9b4d9825b6f admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:41.973500 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:41.985546 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1151/2278] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:21:41 2022] PUT /identity/v3/projects/1885171db7894a6497ded7c75e63ae52/users/42c3e53f1af4451498a55a591762517b/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:41.987531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba2501d-0d7d-4289-9fa5-d9b4d9825b6f admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:41.995596 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1128/2279] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:21:41 2022] POST /identity/v3/users => generated 300 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:21:41.999035 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc11eb-23d5-4c30-8cc4-6f0a5655092b admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:42.010299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc11eb-23d5-4c30-8cc4-6f0a5655092b admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:42.048674 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:42.062378 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba2501d-0d7d-4289-9fa5-d9b4d9825b6f admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:42.068383 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95c84c-37ad-41b9-9925-c38e0138798c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:42.069070 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95c84c-37ad-41b9-9925-c38e0138798c admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:42.069377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95c84c-37ad-41b9-9925-c38e0138798c admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:42.069652 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95c84c-37ad-41b9-9925-c38e0138798c admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:42.070482 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95c84c-37ad-41b9-9925-c38e0138798c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:42.072292 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95c84c-37ad-41b9-9925-c38e0138798c admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:42.078620 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c92e7-2a8b-4ead-aba3-132a8ea70841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:42.079256 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c92e7-2a8b-4ead-aba3-132a8ea70841 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:42.079559 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c92e7-2a8b-4ead-aba3-132a8ea70841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:42.079908 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c92e7-2a8b-4ead-aba3-132a8ea70841 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:42.080518 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930c92e7-2a8b-4ead-aba3-132a8ea70841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:42.082133 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930c92e7-2a8b-4ead-aba3-132a8ea70841 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:42.083231 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1152/2280] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:21:42.084156 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95c84c-37ad-41b9-9925-c38e0138798c admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:42.094389 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95c84c-37ad-41b9-9925-c38e0138798c admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:42.095064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1129/2281] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:21:42.100324 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04213e7-4b53-4550-80c3-584fd0e72a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:42.100538 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c92e7-2a8b-4ead-aba3-132a8ea70841 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:42.101310 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04213e7-4b53-4550-80c3-584fd0e72a3c admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:42.101591 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04213e7-4b53-4550-80c3-584fd0e72a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:42.101863 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04213e7-4b53-4550-80c3-584fd0e72a3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:42.102517 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04213e7-4b53-4550-80c3-584fd0e72a3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:42.104217 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04213e7-4b53-4550-80c3-584fd0e72a3c admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:42.113437 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c92e7-2a8b-4ead-aba3-132a8ea70841 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:42.119192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1153/2282] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:21:42.119192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04213e7-4b53-4550-80c3-584fd0e72a3c admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:42.121753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09828a8-0ac8-424f-8732-11cb6aae963d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:42.122437 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09828a8-0ac8-424f-8732-11cb6aae963d admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:42.122722 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09828a8-0ac8-424f-8732-11cb6aae963d admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:42.122993 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09828a8-0ac8-424f-8732-11cb6aae963d admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:42.123814 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09828a8-0ac8-424f-8732-11cb6aae963d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:42.125436 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09828a8-0ac8-424f-8732-11cb6aae963d admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:42.128455 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04213e7-4b53-4550-80c3-584fd0e72a3c admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:42.133799 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0de8f-6600-4e3e-a4ef-495c2e19c930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:42.134480 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0de8f-6600-4e3e-a4ef-495c2e19c930 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:42.134758 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0de8f-6600-4e3e-a4ef-495c2e19c930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:42.135029 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0de8f-6600-4e3e-a4ef-495c2e19c930 admin admin] PATH_INFO: `/v3/projects/1885171db7894a6497ded7c75e63ae52/users/42c3e53f1af4451498a55a591762517b/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:42.137011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf0de8f-6600-4e3e-a4ef-495c2e19c930 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1885171db7894a6497ded7c75e63ae52, user_id=42c3e53f1af4451498a55a591762517b, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:42.137935 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1130/2283] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:21:42.138914 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09828a8-0ac8-424f-8732-11cb6aae963d admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:42.139739 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf0de8f-6600-4e3e-a4ef-495c2e19c930 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:42.151571 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09828a8-0ac8-424f-8732-11cb6aae963d admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:42.152010 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1154/2284] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:21:42 2022] PUT /identity/v3/projects/1885171db7894a6497ded7c75e63ae52/users/42c3e53f1af4451498a55a591762517b/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:42.152688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0de8f-6600-4e3e-a4ef-495c2e19c930 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:42.228610 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0de8f-6600-4e3e-a4ef-495c2e19c930 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:42.236360 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7368906-55cd-4f02-afc9-01bd50ed7273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:42.237974 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7368906-55cd-4f02-afc9-01bd50ed7273 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:42.238443 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7368906-55cd-4f02-afc9-01bd50ed7273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:42.238913 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7368906-55cd-4f02-afc9-01bd50ed7273 admin admin] PATH_INFO: `/v3/projects/328457097fed40a6b9986d414e4fad7a/users/5205ad3a6e4f46af92c2bff3a14255b4/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:42.244616 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e1b723-1f5e-48be-8dcd-6036a92c431a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:42.246793 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1b723-1f5e-48be-8dcd-6036a92c431a admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:42.247240 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1b723-1f5e-48be-8dcd-6036a92c431a admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:42.247686 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1b723-1f5e-48be-8dcd-6036a92c431a admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:42.248761 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e1b723-1f5e-48be-8dcd-6036a92c431a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:42.251485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e1b723-1f5e-48be-8dcd-6036a92c431a admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:42.269881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1131/2285] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:21:42.270127 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7368906-55cd-4f02-afc9-01bd50ed7273 admin admin] RBAC: Authorizing `identity:create_grant(project_id=328457097fed40a6b9986d414e4fad7a, user_id=5205ad3a6e4f46af92c2bff3a14255b4, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:42.270222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c7e52-9dd2-424e-a242-c19c65bbe680 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:42.270222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c7e52-9dd2-424e-a242-c19c65bbe680 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:42.270222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c7e52-9dd2-424e-a242-c19c65bbe680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:42.272703 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7368906-55cd-4f02-afc9-01bd50ed7273 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:42.300909 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1155/2286] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:21:42 2022] PUT /identity/v3/projects/328457097fed40a6b9986d414e4fad7a/users/5205ad3a6e4f46af92c2bff3a14255b4/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:42.300909 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7368906-55cd-4f02-afc9-01bd50ed7273 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:42.315428 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-4b4c7e52-9dd2-424e-a242-c19c65bbe680 None None] MFA Rules not processed for user `42c3e53f1af4451498a55a591762517b`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:42.316910 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-4b4c7e52-9dd2-424e-a242-c19c65bbe680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:42.408932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:42.423880 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7368906-55cd-4f02-afc9-01bd50ed7273 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:42.424822 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1132/2287] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:42.427906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70ccec-fefd-4507-9426-d6de8769abbb None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:42.428813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70ccec-fefd-4507-9426-d6de8769abbb None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:42.429405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70ccec-fefd-4507-9426-d6de8769abbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:42.430938 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afbff1a-c1e0-4ac0-8aa2-eb15bb21e269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:42.431676 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afbff1a-c1e0-4ac0-8aa2-eb15bb21e269 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:42.432360 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afbff1a-c1e0-4ac0-8aa2-eb15bb21e269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:42.432360 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afbff1a-c1e0-4ac0-8aa2-eb15bb21e269 admin admin] PATH_INFO: `/v3/projects/fbeaf008ab494ebf9fa44bb9c416ffc8/users/a0467e867cdd498f8ede409e44b0c96d/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:42.451552 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afbff1a-c1e0-4ac0-8aa2-eb15bb21e269 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbeaf008ab494ebf9fa44bb9c416ffc8, user_id=a0467e867cdd498f8ede409e44b0c96d, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:42.454318 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afbff1a-c1e0-4ac0-8aa2-eb15bb21e269 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:42.493588 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1156/2288] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:21:42 2022] PUT /identity/v3/projects/fbeaf008ab494ebf9fa44bb9c416ffc8/users/a0467e867cdd498f8ede409e44b0c96d/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:42.493834 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-9a70ccec-fefd-4507-9426-d6de8769abbb None None] MFA Rules not processed for user `5205ad3a6e4f46af92c2bff3a14255b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:42.493834 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-9a70ccec-fefd-4507-9426-d6de8769abbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:42.494357 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afbff1a-c1e0-4ac0-8aa2-eb15bb21e269 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:42.494357 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2afbff1a-c1e0-4ac0-8aa2-eb15bb21e269 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:42.533012 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:42.579731 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afbff1a-c1e0-4ac0-8aa2-eb15bb21e269 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:42.586174 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1133/2289] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:42.586174 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70ccec-fefd-4507-9426-d6de8769abbb None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:42.586174 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-9a70ccec-fefd-4507-9426-d6de8769abbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:42.593217 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade3bdd-0253-455d-8414-d010440f5b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:42.594080 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade3bdd-0253-455d-8414-d010440f5b3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:42.594607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade3bdd-0253-455d-8414-d010440f5b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:42.595125 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade3bdd-0253-455d-8414-d010440f5b3e admin admin] PATH_INFO: `/v3/users/e2ff5514ceba4949bbadf499927954cf` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:42.596701 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade3bdd-0253-455d-8414-d010440f5b3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2ff5514ceba4949bbadf499927954cf)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:42.598978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade3bdd-0253-455d-8414-d010440f5b3e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:42.660078 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0ade3bdd-0253-455d-8414-d010440f5b3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2ff5514ceba4949bbadf499927954cf'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:42.664833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70ccec-fefd-4507-9426-d6de8769abbb None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:42.668668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac4b586-0aa4-43ec-9977-217619148210 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:42.669288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4b586-0aa4-43ec-9977-217619148210 service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:42.669568 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4b586-0aa4-43ec-9977-217619148210 service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:42.669838 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4b586-0aa4-43ec-9977-217619148210 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:42.671325 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0ade3bdd-0253-455d-8414-d010440f5b3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2ff5514ceba4949bbadf499927954cf'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:42.671834 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-8ac4b586-0aa4-43ec-9977-217619148210 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:42.672521 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0ade3bdd-0253-455d-8414-d010440f5b3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2ff5514ceba4949bbadf499927954cf'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:42.673413 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0ade3bdd-0253-455d-8414-d010440f5b3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2ff5514ceba4949bbadf499927954cf'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:42.688789 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0ade3bdd-0253-455d-8414-d010440f5b3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2ff5514ceba4949bbadf499927954cf'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:42.705615 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1157/2290] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:21:42 2022] DELETE /identity/v3/users/e2ff5514ceba4949bbadf499927954cf => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:42.709097 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3d803-dcc9-4fcf-b517-2540d0c303b1 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:42.709388 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3d803-dcc9-4fcf-b517-2540d0c303b1 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:42.709671 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3d803-dcc9-4fcf-b517-2540d0c303b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:42.740723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-78f3d803-dcc9-4fcf-b517-2540d0c303b1 None None] MFA Rules not processed for user `a0467e867cdd498f8ede409e44b0c96d`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:42.741946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-78f3d803-dcc9-4fcf-b517-2540d0c303b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:42.747602 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac4b586-0aa4-43ec-9977-217619148210 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:42.749058 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac4b586-0aa4-43ec-9977-217619148210 service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:42.758588 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1134/2291] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:21:42.759428 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac4b586-0aa4-43ec-9977-217619148210 service cinder] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:42.760758 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-8ac4b586-0aa4-43ec-9977-217619148210 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:42.816516 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:42.818503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1158/2292] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:42 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:42.820693 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d5d4d-c2c3-4863-84c5-969bf115edbf None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:42.821153 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d5d4d-c2c3-4863-84c5-969bf115edbf None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:42.821278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d5d4d-c2c3-4863-84c5-969bf115edbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:42.837411 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac4b586-0aa4-43ec-9977-217619148210 service cinder] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:42.845360 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-9b9d5d4d-c2c3-4863-84c5-969bf115edbf None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:42.847138 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9b9d5d4d-c2c3-4863-84c5-969bf115edbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:42.888924 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:42.894227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e772f52-1f0d-4263-b2ca-f50ddc9d8639 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed75c0be809d4a0eb0e2ccc59805b1ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:42.894957 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e772f52-1f0d-4263-b2ca-f50ddc9d8639 service glance] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:42.895341 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e772f52-1f0d-4263-b2ca-f50ddc9d8639 service glance] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:42.895703 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e772f52-1f0d-4263-b2ca-f50ddc9d8639 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:42.898495 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-6e772f52-1f0d-4263-b2ca-f50ddc9d8639 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:42.931773 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1159/2293] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:42.932797 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d5d4d-c2c3-4863-84c5-969bf115edbf None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:42.934312 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9b9d5d4d-c2c3-4863-84c5-969bf115edbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:42.984132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e772f52-1f0d-4263-b2ca-f50ddc9d8639 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:42.985799 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e772f52-1f0d-4263-b2ca-f50ddc9d8639 service glance] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:42.998859 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1135/2294] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:21:42.999786 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e772f52-1f0d-4263-b2ca-f50ddc9d8639 service glance] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:43.001183 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-6e772f52-1f0d-4263-b2ca-f50ddc9d8639 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:43.018904 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d5d4d-c2c3-4863-84c5-969bf115edbf None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:43.024219 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f60c7-ffde-49c4-a7e8-354a94a07885 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:43.027289 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f60c7-ffde-49c4-a7e8-354a94a07885 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:43.027654 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f60c7-ffde-49c4-a7e8-354a94a07885 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:43.027932 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f60c7-ffde-49c4-a7e8-354a94a07885 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:43.041312 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f60c7-ffde-49c4-a7e8-354a94a07885 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:43.044979 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f60c7-ffde-49c4-a7e8-354a94a07885 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:43.056888 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1160/2295] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:42 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:21:43.093205 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e772f52-1f0d-4263-b2ca-f50ddc9d8639 service glance] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:43.102562 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f96b6-f6ed-4229-be7d-68afead1f36b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:43.103949 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f96b6-f6ed-4229-be7d-68afead1f36b service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:43.104288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f96b6-f6ed-4229-be7d-68afead1f36b service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:43.105025 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f96b6-f6ed-4229-be7d-68afead1f36b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:43.160740 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f96b6-f6ed-4229-be7d-68afead1f36b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:43.165473 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f96b6-f6ed-4229-be7d-68afead1f36b service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:43.210251 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1136/2296] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:21:43.660724 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f60c7-ffde-49c4-a7e8-354a94a07885 service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:43.662290 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-452f60c7-ffde-49c4-a7e8-354a94a07885 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:43.725682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f60c7-ffde-49c4-a7e8-354a94a07885 service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:43.742792 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c5da2-d058-4e09-9b9d-d76ac4d213a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:43.742792 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c5da2-d058-4e09-9b9d-d76ac4d213a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:43.743083 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c5da2-d058-4e09-9b9d-d76ac4d213a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:43.743339 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c5da2-d058-4e09-9b9d-d76ac4d213a4 admin admin] PATH_INFO: `/v3/projects/ae0e87fd214d43eea687f00ddfc9d97c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:43.744805 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684c5da2-d058-4e09-9b9d-d76ac4d213a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae0e87fd214d43eea687f00ddfc9d97c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:43.748121 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684c5da2-d058-4e09-9b9d-d76ac4d213a4 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:43.885286 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-684c5da2-d058-4e09-9b9d-d76ac4d213a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae0e87fd214d43eea687f00ddfc9d97c', 'request_id': 'req-684c5da2-d058-4e09-9b9d-d76ac4d213a4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:43.894398 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-684c5da2-d058-4e09-9b9d-d76ac4d213a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae0e87fd214d43eea687f00ddfc9d97c', 'request_id': 'req-684c5da2-d058-4e09-9b9d-d76ac4d213a4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:43.904440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-684c5da2-d058-4e09-9b9d-d76ac4d213a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae0e87fd214d43eea687f00ddfc9d97c', 'request_id': 'req-684c5da2-d058-4e09-9b9d-d76ac4d213a4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:43.932793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-684c5da2-d058-4e09-9b9d-d76ac4d213a4 admin admin] The token cache is being invalidate because project ae0e87fd214d43eea687f00ddfc9d97c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:21:43.933554 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-684c5da2-d058-4e09-9b9d-d76ac4d213a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:43.938429 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-684c5da2-d058-4e09-9b9d-d76ac4d213a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:21:43.946722 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1161/2297] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:21:43 2022] DELETE /identity/v3/projects/ae0e87fd214d43eea687f00ddfc9d97c => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:21:43.948554 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7fbea-f964-4e01-8ab9-2d84440e2a0a None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:43.949040 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7fbea-f964-4e01-8ab9-2d84440e2a0a None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:43.949268 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7fbea-f964-4e01-8ab9-2d84440e2a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:43.984348 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-bfe7fbea-f964-4e01-8ab9-2d84440e2a0a None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:43.985605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-bfe7fbea-f964-4e01-8ab9-2d84440e2a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:44.048535 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:44.066889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1137/2298] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:21:44.076181 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462c615-e71c-4929-98a3-c52e9f906d19 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:44.077371 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462c615-e71c-4929-98a3-c52e9f906d19 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:44.077706 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462c615-e71c-4929-98a3-c52e9f906d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:44.100684 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-8462c615-e71c-4929-98a3-c52e9f906d19 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:44.102190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8462c615-e71c-4929-98a3-c52e9f906d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:44.112255 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1162/2299] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:44.117014 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7fbea-f964-4e01-8ab9-2d84440e2a0a None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:44.130214 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7fbea-f964-4e01-8ab9-2d84440e2a0a None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:44.138236 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e77745-eca4-41f4-8306-cedc22a5386f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:44.139142 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e77745-eca4-41f4-8306-cedc22a5386f admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:44.139682 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e77745-eca4-41f4-8306-cedc22a5386f admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:44.140160 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e77745-eca4-41f4-8306-cedc22a5386f admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:44.141228 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e77745-eca4-41f4-8306-cedc22a5386f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:21:44.141633 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e77745-eca4-41f4-8306-cedc22a5386f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:44.143886 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e77745-eca4-41f4-8306-cedc22a5386f admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:44.164646 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1138/2300] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:21:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:21:44.169921 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462c615-e71c-4929-98a3-c52e9f906d19 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:44.184745 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462c615-e71c-4929-98a3-c52e9f906d19 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:44.191915 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae65c0-4e83-499a-af97-967effd4d3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:44.192716 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae65c0-4e83-499a-af97-967effd4d3f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:44.193233 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae65c0-4e83-499a-af97-967effd4d3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:44.193668 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae65c0-4e83-499a-af97-967effd4d3f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:44.194561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fae65c0-4e83-499a-af97-967effd4d3f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:44.197552 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fae65c0-4e83-499a-af97-967effd4d3f4 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:44.276620 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1163/2301] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:21:44 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:21:44.284704 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e77745-eca4-41f4-8306-cedc22a5386f admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:44.326965 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e77745-eca4-41f4-8306-cedc22a5386f admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:44.484551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b06245-b1c1-4e3d-b32d-009180fc11de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:44.485107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b06245-b1c1-4e3d-b32d-009180fc11de admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:44.485382 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b06245-b1c1-4e3d-b32d-009180fc11de admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:44.485604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b06245-b1c1-4e3d-b32d-009180fc11de admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:44.486317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b06245-b1c1-4e3d-b32d-009180fc11de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:44.488915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b06245-b1c1-4e3d-b32d-009180fc11de admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:44.513544 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1139/2302] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:21:44 2022] POST /identity/v3/users => generated 292 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:21:44.519146 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae65c0-4e83-499a-af97-967effd4d3f4 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:44.532646 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae65c0-4e83-499a-af97-967effd4d3f4 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:44.543311 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f6847-7aed-4ea6-bb3c-c9b433735ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:44.544272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f6847-7aed-4ea6-bb3c-c9b433735ad0 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:44.544720 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f6847-7aed-4ea6-bb3c-c9b433735ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:44.545107 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f6847-7aed-4ea6-bb3c-c9b433735ad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:44.545904 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f6847-7aed-4ea6-bb3c-c9b433735ad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:44.548004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f6847-7aed-4ea6-bb3c-c9b433735ad0 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:44.550633 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b06245-b1c1-4e3d-b32d-009180fc11de admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:44.552927 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-94b06245-b1c1-4e3d-b32d-009180fc11de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:44.563575 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1164/2303] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:21:44.566179 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f6847-7aed-4ea6-bb3c-c9b433735ad0 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:44.576362 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f6847-7aed-4ea6-bb3c-c9b433735ad0 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:44.581892 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04495f63-a8a1-4c00-b410-05d5627cf0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:44.582837 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04495f63-a8a1-4c00-b410-05d5627cf0e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:44.582964 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04495f63-a8a1-4c00-b410-05d5627cf0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:44.583234 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04495f63-a8a1-4c00-b410-05d5627cf0e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:44.583976 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04495f63-a8a1-4c00-b410-05d5627cf0e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:44.585633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04495f63-a8a1-4c00-b410-05d5627cf0e2 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:44.597010 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1165/2304] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:21:44.603049 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04495f63-a8a1-4c00-b410-05d5627cf0e2 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:44.615025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04495f63-a8a1-4c00-b410-05d5627cf0e2 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:44.622618 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979315c-33a4-469e-a3eb-92d1133f0598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:44.623624 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979315c-33a4-469e-a3eb-92d1133f0598 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:44.624034 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979315c-33a4-469e-a3eb-92d1133f0598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:44.624575 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979315c-33a4-469e-a3eb-92d1133f0598 admin admin] PATH_INFO: `/v3/projects/57a93fc888a145c88b28b8b73060731e/users/4e187162f3e64acdb8cd24f660ceeca6/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:44.626260 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b06245-b1c1-4e3d-b32d-009180fc11de admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:44.630341 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe0d80c-2d3c-4b6b-a327-219cc2fa1448 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:44.630965 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0d80c-2d3c-4b6b-a327-219cc2fa1448 service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:44.631241 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0d80c-2d3c-4b6b-a327-219cc2fa1448 service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:44.631513 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0d80c-2d3c-4b6b-a327-219cc2fa1448 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:44.633390 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-7fe0d80c-2d3c-4b6b-a327-219cc2fa1448 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:44.640036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:44.646722 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979315c-33a4-469e-a3eb-92d1133f0598 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57a93fc888a145c88b28b8b73060731e, user_id=4e187162f3e64acdb8cd24f660ceeca6, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:44.650883 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979315c-33a4-469e-a3eb-92d1133f0598 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:44.669941 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1166/2305] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:21:44 2022] PUT /identity/v3/projects/57a93fc888a145c88b28b8b73060731e/users/4e187162f3e64acdb8cd24f660ceeca6/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:21:44.676286 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1c705-f1d2-4292-b7cf-cfa95cd5f166 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:44.676690 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1c705-f1d2-4292-b7cf-cfa95cd5f166 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:44.677149 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1c705-f1d2-4292-b7cf-cfa95cd5f166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:44.699641 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-f5d1c705-f1d2-4292-b7cf-cfa95cd5f166 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:44.700913 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f5d1c705-f1d2-4292-b7cf-cfa95cd5f166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:44.718802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe0d80c-2d3c-4b6b-a327-219cc2fa1448 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:44.720414 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe0d80c-2d3c-4b6b-a327-219cc2fa1448 service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:44.733943 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1140/2306] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:21:44.839205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1167/2307] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:44.842414 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe0d80c-2d3c-4b6b-a327-219cc2fa1448 service cinder] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:44.843916 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-7fe0d80c-2d3c-4b6b-a327-219cc2fa1448 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:44.913371 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:44.960469 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe0d80c-2d3c-4b6b-a327-219cc2fa1448 service cinder] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:45.056973 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a8f95-5171-4fc1-9f1d-3c830939796b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:45.058027 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a8f95-5171-4fc1-9f1d-3c830939796b service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:45.058535 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a8f95-5171-4fc1-9f1d-3c830939796b service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:45.059029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a8f95-5171-4fc1-9f1d-3c830939796b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:45.078090 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8a8f95-5171-4fc1-9f1d-3c830939796b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:45.081634 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8a8f95-5171-4fc1-9f1d-3c830939796b service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:45.096490 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1141/2308] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:21:46.009716 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c2874-a3ee-4e74-9298-a5b24bdb3c69 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:46.010897 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c2874-a3ee-4e74-9298-a5b24bdb3c69 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:46.012451 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c2874-a3ee-4e74-9298-a5b24bdb3c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:46.051220 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-4d9c2874-a3ee-4e74-9298-a5b24bdb3c69 None None] MFA Rules not processed for user `4e187162f3e64acdb8cd24f660ceeca6`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:46.052919 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4d9c2874-a3ee-4e74-9298-a5b24bdb3c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:46.194689 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1168/2309] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:46.209907 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a8f95-5171-4fc1-9f1d-3c830939796b service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:46.212318 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-1e8a8f95-5171-4fc1-9f1d-3c830939796b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:46.307645 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a8f95-5171-4fc1-9f1d-3c830939796b service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:46.311744 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c3dff-a24b-426b-83ae-32a4aa2b5a5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:46.312415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c3dff-a24b-426b-83ae-32a4aa2b5a5b service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:46.312694 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c3dff-a24b-426b-83ae-32a4aa2b5a5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:46.312965 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c3dff-a24b-426b-83ae-32a4aa2b5a5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:46.324150 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c3dff-a24b-426b-83ae-32a4aa2b5a5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:46.327258 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c3dff-a24b-426b-83ae-32a4aa2b5a5b service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:46.339857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1142/2310] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:21:48.483043 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c2874-a3ee-4e74-9298-a5b24bdb3c69 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:48.501407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c2874-a3ee-4e74-9298-a5b24bdb3c69 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:48.523445 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fbb012-fec5-44a1-8597-6dbd32592908 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:48.524823 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fbb012-fec5-44a1-8597-6dbd32592908 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:48.525336 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fbb012-fec5-44a1-8597-6dbd32592908 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:48.525844 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fbb012-fec5-44a1-8597-6dbd32592908 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:48.528958 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d7fbb012-fec5-44a1-8597-6dbd32592908 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:48.615696 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fbb012-fec5-44a1-8597-6dbd32592908 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:48.617046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fbb012-fec5-44a1-8597-6dbd32592908 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:48.628630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1169/2311] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:48 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:21:51.135598 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93f76b-3942-46d3-9c02-1811eacf5967 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:51.136481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93f76b-3942-46d3-9c02-1811eacf5967 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:51.136770 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93f76b-3942-46d3-9c02-1811eacf5967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:51.160231 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-8b93f76b-3942-46d3-9c02-1811eacf5967 None None] MFA Rules not processed for user `4e187162f3e64acdb8cd24f660ceeca6`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:51.161671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-8b93f76b-3942-46d3-9c02-1811eacf5967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:51.169333 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1143/2312] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:21:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:55.687362 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fbb012-fec5-44a1-8597-6dbd32592908 service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:55.688990 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d7fbb012-fec5-44a1-8597-6dbd32592908 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:55.703997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:55.769499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fbb012-fec5-44a1-8597-6dbd32592908 service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:55.776930 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552f0ba-7bb4-49b7-91e9-bbc331eefcec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524bcfffcdfa4790885e0d4cbd66fa80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:55.777591 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552f0ba-7bb4-49b7-91e9-bbc331eefcec service placement] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:55.778079 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552f0ba-7bb4-49b7-91e9-bbc331eefcec service placement] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:55.778560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552f0ba-7bb4-49b7-91e9-bbc331eefcec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:55.780833 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-0552f0ba-7bb4-49b7-91e9-bbc331eefcec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:55.794547 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0552f0ba-7bb4-49b7-91e9-bbc331eefcec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:55.796437 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0552f0ba-7bb4-49b7-91e9-bbc331eefcec service placement] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:55.807822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1170/2313] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:21:59.916104 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93f76b-3942-46d3-9c02-1811eacf5967 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:59.917594 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-8b93f76b-3942-46d3-9c02-1811eacf5967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:59.941992 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93f76b-3942-46d3-9c02-1811eacf5967 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:59.947719 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bce983-e39d-4a29-bbb9-e2ebab5ab2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:59.948387 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bce983-e39d-4a29-bbb9-e2ebab5ab2cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:59.948680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bce983-e39d-4a29-bbb9-e2ebab5ab2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:59.948954 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bce983-e39d-4a29-bbb9-e2ebab5ab2cb admin admin] PATH_INFO: `/v3/users/a0467e867cdd498f8ede409e44b0c96d` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:59.950318 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bce983-e39d-4a29-bbb9-e2ebab5ab2cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0467e867cdd498f8ede409e44b0c96d)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:59.952605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bce983-e39d-4a29-bbb9-e2ebab5ab2cb admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:00.064097 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-c2bce983-e39d-4a29-bbb9-e2ebab5ab2cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0467e867cdd498f8ede409e44b0c96d'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:00.079632 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-c2bce983-e39d-4a29-bbb9-e2ebab5ab2cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0467e867cdd498f8ede409e44b0c96d'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:00.080923 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-c2bce983-e39d-4a29-bbb9-e2ebab5ab2cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0467e867cdd498f8ede409e44b0c96d'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:00.108733 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-c2bce983-e39d-4a29-bbb9-e2ebab5ab2cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0467e867cdd498f8ede409e44b0c96d'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:00.115303 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:00.126242 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-c2bce983-e39d-4a29-bbb9-e2ebab5ab2cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0467e867cdd498f8ede409e44b0c96d'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:00.135900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1144/2314] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:21:59 2022] DELETE /identity/v3/users/a0467e867cdd498f8ede409e44b0c96d => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:22:00.142384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d368b-222b-4355-aa84-c11dc1a325c2 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:00.143043 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d368b-222b-4355-aa84-c11dc1a325c2 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:00.143657 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d368b-222b-4355-aa84-c11dc1a325c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:00.169207 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-2b7d368b-222b-4355-aa84-c11dc1a325c2 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:00.170854 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2b7d368b-222b-4355-aa84-c11dc1a325c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:00.268423 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1171/2315] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:22:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:00.286547 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bce983-e39d-4a29-bbb9-e2ebab5ab2cb admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:00.289552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-c2bce983-e39d-4a29-bbb9-e2ebab5ab2cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:00.373079 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bce983-e39d-4a29-bbb9-e2ebab5ab2cb admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:00.377888 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f4ca4-1970-4b30-8769-17d83f528a22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:00.379570 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f4ca4-1970-4b30-8769-17d83f528a22 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:00.379570 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f4ca4-1970-4b30-8769-17d83f528a22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:00.379924 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f4ca4-1970-4b30-8769-17d83f528a22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:00.395828 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f4ca4-1970-4b30-8769-17d83f528a22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:00.397729 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f4ca4-1970-4b30-8769-17d83f528a22 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:00.415045 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1145/2316] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:22:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:22:00.879033 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d368b-222b-4355-aa84-c11dc1a325c2 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:00.881496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2b7d368b-222b-4355-aa84-c11dc1a325c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:00.928924 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d368b-222b-4355-aa84-c11dc1a325c2 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:00.937236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179459e9-c001-4698-9dcc-9400f40d0048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:00.937997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179459e9-c001-4698-9dcc-9400f40d0048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:00.938319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179459e9-c001-4698-9dcc-9400f40d0048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:00.938590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179459e9-c001-4698-9dcc-9400f40d0048 admin admin] PATH_INFO: `/v3/projects/fbeaf008ab494ebf9fa44bb9c416ffc8` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:00.939906 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179459e9-c001-4698-9dcc-9400f40d0048 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbeaf008ab494ebf9fa44bb9c416ffc8)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:00.942465 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179459e9-c001-4698-9dcc-9400f40d0048 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:01.013683 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-179459e9-c001-4698-9dcc-9400f40d0048 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbeaf008ab494ebf9fa44bb9c416ffc8', 'request_id': 'req-179459e9-c001-4698-9dcc-9400f40d0048'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:01.020692 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-179459e9-c001-4698-9dcc-9400f40d0048 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbeaf008ab494ebf9fa44bb9c416ffc8', 'request_id': 'req-179459e9-c001-4698-9dcc-9400f40d0048'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:01.027854 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-179459e9-c001-4698-9dcc-9400f40d0048 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbeaf008ab494ebf9fa44bb9c416ffc8', 'request_id': 'req-179459e9-c001-4698-9dcc-9400f40d0048'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:01.048426 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-179459e9-c001-4698-9dcc-9400f40d0048 admin admin] The token cache is being invalidate because project fbeaf008ab494ebf9fa44bb9c416ffc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:22:01.048945 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-179459e9-c001-4698-9dcc-9400f40d0048 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:01.050662 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-179459e9-c001-4698-9dcc-9400f40d0048 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:01.053916 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1172/2317] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:22:00 2022] DELETE /identity/v3/projects/fbeaf008ab494ebf9fa44bb9c416ffc8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:22:01.060544 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35220ea-66d6-4051-b52f-1aca7b844b2d None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:01.061306 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35220ea-66d6-4051-b52f-1aca7b844b2d None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:01.061306 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35220ea-66d6-4051-b52f-1aca7b844b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:01.103415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-e35220ea-66d6-4051-b52f-1aca7b844b2d None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:01.104731 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-e35220ea-66d6-4051-b52f-1aca7b844b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:01.207821 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1146/2318] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:22:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:01.216922 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd554f25-4947-4cbb-9a17-409ed3c4cd76 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:01.217867 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd554f25-4947-4cbb-9a17-409ed3c4cd76 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:01.218050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd554f25-4947-4cbb-9a17-409ed3c4cd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:01.240345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-dd554f25-4947-4cbb-9a17-409ed3c4cd76 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:01.241709 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-dd554f25-4947-4cbb-9a17-409ed3c4cd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:01.252907 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1173/2319] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:22:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:01.259827 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35220ea-66d6-4051-b52f-1aca7b844b2d None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:01.270675 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35220ea-66d6-4051-b52f-1aca7b844b2d None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:01.275907 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ffa07f-4dd0-49a4-8c11-fecf3dc41566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:01.276531 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ffa07f-4dd0-49a4-8c11-fecf3dc41566 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:01.276817 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ffa07f-4dd0-49a4-8c11-fecf3dc41566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:01.277082 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ffa07f-4dd0-49a4-8c11-fecf3dc41566 admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:01.277813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ffa07f-4dd0-49a4-8c11-fecf3dc41566 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:22:01.278123 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ffa07f-4dd0-49a4-8c11-fecf3dc41566 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:01.279782 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ffa07f-4dd0-49a4-8c11-fecf3dc41566 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:01.299103 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1147/2320] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:22:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:22:01.303399 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd554f25-4947-4cbb-9a17-409ed3c4cd76 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:01.318057 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd554f25-4947-4cbb-9a17-409ed3c4cd76 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:01.323584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59e686-9a13-4c86-ab99-72e0b3bc0337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:01.324337 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59e686-9a13-4c86-ab99-72e0b3bc0337 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:01.324658 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59e686-9a13-4c86-ab99-72e0b3bc0337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:01.324970 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59e686-9a13-4c86-ab99-72e0b3bc0337 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:01.325715 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a59e686-9a13-4c86-ab99-72e0b3bc0337 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:01.327796 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a59e686-9a13-4c86-ab99-72e0b3bc0337 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:01.352763 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:01.363768 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1174/2321] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:22:01 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:22:01.370360 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ffa07f-4dd0-49a4-8c11-fecf3dc41566 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:01.384586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ffa07f-4dd0-49a4-8c11-fecf3dc41566 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:01.451387 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09635387-306e-4aaa-8bc1-9ff880429374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:01.452050 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09635387-306e-4aaa-8bc1-9ff880429374 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:01.452335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09635387-306e-4aaa-8bc1-9ff880429374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:01.452605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09635387-306e-4aaa-8bc1-9ff880429374 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:01.453271 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09635387-306e-4aaa-8bc1-9ff880429374 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:01.455232 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09635387-306e-4aaa-8bc1-9ff880429374 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:01.485523 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1148/2322] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:22:01 2022] POST /identity/v3/users => generated 287 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:22:01.491589 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59e686-9a13-4c86-ab99-72e0b3bc0337 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:01.508974 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59e686-9a13-4c86-ab99-72e0b3bc0337 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:01.516669 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9197f-f4f0-4d3a-8850-e03b512fec2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:01.517435 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9197f-f4f0-4d3a-8850-e03b512fec2f admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:01.518168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9197f-f4f0-4d3a-8850-e03b512fec2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:01.518585 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9197f-f4f0-4d3a-8850-e03b512fec2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:01.519810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a9197f-f4f0-4d3a-8850-e03b512fec2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:01.523269 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a9197f-f4f0-4d3a-8850-e03b512fec2f admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:01.536579 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1175/2323] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:22:01 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:22:01.541633 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09635387-306e-4aaa-8bc1-9ff880429374 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:01.556106 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09635387-306e-4aaa-8bc1-9ff880429374 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:01.561986 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e530a-2faa-44ce-8e87-5166a21927e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:01.562784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e530a-2faa-44ce-8e87-5166a21927e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:01.563127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e530a-2faa-44ce-8e87-5166a21927e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:01.563276 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e530a-2faa-44ce-8e87-5166a21927e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:01.563924 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e530a-2faa-44ce-8e87-5166a21927e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:01.565550 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e530a-2faa-44ce-8e87-5166a21927e3 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:01.572850 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:01.585814 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1149/2324] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:22:01 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:22:01.586779 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9197f-f4f0-4d3a-8850-e03b512fec2f admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:01.593529 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e530a-2faa-44ce-8e87-5166a21927e3 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:01.595550 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-815e530a-2faa-44ce-8e87-5166a21927e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:01.599914 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9197f-f4f0-4d3a-8850-e03b512fec2f admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:01.605006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0127b63c-04df-4eef-8977-cea8fa8bca71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:01.605613 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127b63c-04df-4eef-8977-cea8fa8bca71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:01.605898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127b63c-04df-4eef-8977-cea8fa8bca71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:01.606170 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127b63c-04df-4eef-8977-cea8fa8bca71 admin admin] PATH_INFO: `/v3/projects/88e8ed647b3845119144655e27322ae1/users/8ff56403efde4509bc830967a1c25ccc/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:01.613794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e530a-2faa-44ce-8e87-5166a21927e3 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:01.619541 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a02816-3871-480c-9515-11a238c92677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:01.621358 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a02816-3871-480c-9515-11a238c92677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:01.621358 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a02816-3871-480c-9515-11a238c92677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:01.621358 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a02816-3871-480c-9515-11a238c92677 admin admin] PATH_INFO: `/v3/users/dba21c47c98441dd857160264520dc9d` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:01.621978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a02816-3871-480c-9515-11a238c92677 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dba21c47c98441dd857160264520dc9d)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:01.623975 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a02816-3871-480c-9515-11a238c92677 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:01.636191 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0127b63c-04df-4eef-8977-cea8fa8bca71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88e8ed647b3845119144655e27322ae1, user_id=8ff56403efde4509bc830967a1c25ccc, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:01.638983 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0127b63c-04df-4eef-8977-cea8fa8bca71 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:01.663620 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1176/2325] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:22:01 2022] PUT /identity/v3/projects/88e8ed647b3845119144655e27322ae1/users/8ff56403efde4509bc830967a1c25ccc/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:22:01.665692 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff98343e-ef29-4f9f-8d48-9606be7d0d37 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:01.666180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff98343e-ef29-4f9f-8d48-9606be7d0d37 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:01.666634 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff98343e-ef29-4f9f-8d48-9606be7d0d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:01.698123 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-ff98343e-ef29-4f9f-8d48-9606be7d0d37 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:01.699574 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ff98343e-ef29-4f9f-8d48-9606be7d0d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:01.706204 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-90a02816-3871-480c-9515-11a238c92677 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dba21c47c98441dd857160264520dc9d'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:01.716526 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-90a02816-3871-480c-9515-11a238c92677 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dba21c47c98441dd857160264520dc9d'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:01.717442 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-90a02816-3871-480c-9515-11a238c92677 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dba21c47c98441dd857160264520dc9d'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:01.732278 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-90a02816-3871-480c-9515-11a238c92677 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dba21c47c98441dd857160264520dc9d'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:01.744415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-90a02816-3871-480c-9515-11a238c92677 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dba21c47c98441dd857160264520dc9d'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:01.750985 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1150/2326] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:22:01 2022] DELETE /identity/v3/users/dba21c47c98441dd857160264520dc9d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:22:01.755660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab783a5-cbf5-4029-b326-671d762a924c None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:01.755945 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab783a5-cbf5-4029-b326-671d762a924c None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:01.756222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab783a5-cbf5-4029-b326-671d762a924c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:01.776883 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1177/2327] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:22:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:01.777259 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-7ab783a5-cbf5-4029-b326-671d762a924c None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:01.778495 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-7ab783a5-cbf5-4029-b326-671d762a924c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:01.787701 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff98343e-ef29-4f9f-8d48-9606be7d0d37 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:01.789550 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ff98343e-ef29-4f9f-8d48-9606be7d0d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:01.868252 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff98343e-ef29-4f9f-8d48-9606be7d0d37 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:01.870893 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1151/2328] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:22:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:22:01.886705 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab783a5-cbf5-4029-b326-671d762a924c None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:01.942267 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e82785-4dbb-49a8-b6c9-87d79a828f8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:01.943003 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e82785-4dbb-49a8-b6c9-87d79a828f8d service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:01.943368 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e82785-4dbb-49a8-b6c9-87d79a828f8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:01.943730 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e82785-4dbb-49a8-b6c9-87d79a828f8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:01.955871 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e82785-4dbb-49a8-b6c9-87d79a828f8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:01.958834 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e82785-4dbb-49a8-b6c9-87d79a828f8d service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:01.983625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1178/2329] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:22:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:22:01.988992 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab783a5-cbf5-4029-b326-671d762a924c None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:01.992877 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65f9ca-9ff3-4dfc-b687-32da115c7646 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:01.993505 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65f9ca-9ff3-4dfc-b687-32da115c7646 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:01.993785 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65f9ca-9ff3-4dfc-b687-32da115c7646 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:01.994059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65f9ca-9ff3-4dfc-b687-32da115c7646 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:02.004794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65f9ca-9ff3-4dfc-b687-32da115c7646 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:02.006276 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65f9ca-9ff3-4dfc-b687-32da115c7646 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:02.020691 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1152/2330] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:22:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:22:02.800439 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e82785-4dbb-49a8-b6c9-87d79a828f8d service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:02.803823 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-06e82785-4dbb-49a8-b6c9-87d79a828f8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:02.852912 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e82785-4dbb-49a8-b6c9-87d79a828f8d service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:02.863388 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7721220-a6d5-4398-9ca8-b20897a701bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:02.864350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7721220-a6d5-4398-9ca8-b20897a701bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:02.864735 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7721220-a6d5-4398-9ca8-b20897a701bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:02.865267 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7721220-a6d5-4398-9ca8-b20897a701bf admin admin] PATH_INFO: `/v3/projects/e20da78cb53f46a7876263720edb8830` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:02.866880 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7721220-a6d5-4398-9ca8-b20897a701bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=e20da78cb53f46a7876263720edb8830)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:02.869469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7721220-a6d5-4398-9ca8-b20897a701bf admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:02.889916 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:02.941954 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a7721220-a6d5-4398-9ca8-b20897a701bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e20da78cb53f46a7876263720edb8830', 'request_id': 'req-a7721220-a6d5-4398-9ca8-b20897a701bf'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:02.950481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a7721220-a6d5-4398-9ca8-b20897a701bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e20da78cb53f46a7876263720edb8830', 'request_id': 'req-a7721220-a6d5-4398-9ca8-b20897a701bf'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:02.958936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a7721220-a6d5-4398-9ca8-b20897a701bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e20da78cb53f46a7876263720edb8830', 'request_id': 'req-a7721220-a6d5-4398-9ca8-b20897a701bf'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:02.983760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a7721220-a6d5-4398-9ca8-b20897a701bf admin admin] The token cache is being invalidate because project e20da78cb53f46a7876263720edb8830 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:22:02.984397 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a7721220-a6d5-4398-9ca8-b20897a701bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:02.986428 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a7721220-a6d5-4398-9ca8-b20897a701bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:02.989430 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1179/2331] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:22:02 2022] DELETE /identity/v3/projects/e20da78cb53f46a7876263720edb8830 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:22:02.994215 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65f9ca-9ff3-4dfc-b687-32da115c7646 service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:03.005249 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65f9ca-9ff3-4dfc-b687-32da115c7646 service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:03.020514 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:03.164234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449ff5f-2fb7-48da-9388-6c6982f1cf09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:03.164955 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449ff5f-2fb7-48da-9388-6c6982f1cf09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:03.165578 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449ff5f-2fb7-48da-9388-6c6982f1cf09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:03.166263 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449ff5f-2fb7-48da-9388-6c6982f1cf09 admin admin] PATH_INFO: `/v3/users/8ecc620b7e3f4eafbc304ecc0c64ccc0` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:03.168548 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a449ff5f-2fb7-48da-9388-6c6982f1cf09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ecc620b7e3f4eafbc304ecc0c64ccc0)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:03.171047 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a449ff5f-2fb7-48da-9388-6c6982f1cf09 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:03.289062 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-a449ff5f-2fb7-48da-9388-6c6982f1cf09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ecc620b7e3f4eafbc304ecc0c64ccc0'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:03.307848 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-a449ff5f-2fb7-48da-9388-6c6982f1cf09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ecc620b7e3f4eafbc304ecc0c64ccc0'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:03.308839 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-a449ff5f-2fb7-48da-9388-6c6982f1cf09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ecc620b7e3f4eafbc304ecc0c64ccc0'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:03.351474 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-a449ff5f-2fb7-48da-9388-6c6982f1cf09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ecc620b7e3f4eafbc304ecc0c64ccc0'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:03.375463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-a449ff5f-2fb7-48da-9388-6c6982f1cf09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ecc620b7e3f4eafbc304ecc0c64ccc0'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:03.391015 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1153/2332] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:22:02 2022] DELETE /identity/v3/users/8ecc620b7e3f4eafbc304ecc0c64ccc0 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:22:04.157264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7721220-a6d5-4398-9ca8-b20897a701bf admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:04.161692 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a7721220-a6d5-4398-9ca8-b20897a701bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:04.192552 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7721220-a6d5-4398-9ca8-b20897a701bf admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:04.276520 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5894fc3-bd47-4536-9a69-0d5ab180e0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:04.277524 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5894fc3-bd47-4536-9a69-0d5ab180e0af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:04.278012 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5894fc3-bd47-4536-9a69-0d5ab180e0af admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:04.278419 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5894fc3-bd47-4536-9a69-0d5ab180e0af admin admin] PATH_INFO: `/v3/projects/b97a84390f4e437d949342fe466ae89a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:04.280079 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5894fc3-bd47-4536-9a69-0d5ab180e0af admin admin] RBAC: Authorizing `identity:delete_project(project_id=b97a84390f4e437d949342fe466ae89a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:04.283161 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5894fc3-bd47-4536-9a69-0d5ab180e0af admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:04.384144 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b5894fc3-bd47-4536-9a69-0d5ab180e0af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b97a84390f4e437d949342fe466ae89a', 'request_id': 'req-b5894fc3-bd47-4536-9a69-0d5ab180e0af'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:04.406424 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b5894fc3-bd47-4536-9a69-0d5ab180e0af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b97a84390f4e437d949342fe466ae89a', 'request_id': 'req-b5894fc3-bd47-4536-9a69-0d5ab180e0af'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:04.438908 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b5894fc3-bd47-4536-9a69-0d5ab180e0af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b97a84390f4e437d949342fe466ae89a', 'request_id': 'req-b5894fc3-bd47-4536-9a69-0d5ab180e0af'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:04.459774 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b5894fc3-bd47-4536-9a69-0d5ab180e0af admin admin] The token cache is being invalidate because project b97a84390f4e437d949342fe466ae89a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:22:04.459774 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b5894fc3-bd47-4536-9a69-0d5ab180e0af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:04.460691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b5894fc3-bd47-4536-9a69-0d5ab180e0af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:04.462345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1180/2333] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:22:04 2022] DELETE /identity/v3/projects/b97a84390f4e437d949342fe466ae89a => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:22:04.467110 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449ff5f-2fb7-48da-9388-6c6982f1cf09 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:04.473408 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-a449ff5f-2fb7-48da-9388-6c6982f1cf09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:04.500313 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449ff5f-2fb7-48da-9388-6c6982f1cf09 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:04.579476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e112c-cdfc-49ef-a4bb-a48ef2bdd0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:04.580191 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e112c-cdfc-49ef-a4bb-a48ef2bdd0c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:04.580465 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e112c-cdfc-49ef-a4bb-a48ef2bdd0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:04.582030 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e112c-cdfc-49ef-a4bb-a48ef2bdd0c0 admin admin] PATH_INFO: `/v3/users/42c3e53f1af4451498a55a591762517b` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:04.583491 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e112c-cdfc-49ef-a4bb-a48ef2bdd0c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42c3e53f1af4451498a55a591762517b)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:04.585529 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e112c-cdfc-49ef-a4bb-a48ef2bdd0c0 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:04.680537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:04.721638 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-383e112c-cdfc-49ef-a4bb-a48ef2bdd0c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42c3e53f1af4451498a55a591762517b'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:04.739249 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-383e112c-cdfc-49ef-a4bb-a48ef2bdd0c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42c3e53f1af4451498a55a591762517b'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:04.744847 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-383e112c-cdfc-49ef-a4bb-a48ef2bdd0c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42c3e53f1af4451498a55a591762517b'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:04.777204 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-383e112c-cdfc-49ef-a4bb-a48ef2bdd0c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42c3e53f1af4451498a55a591762517b'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:04.801203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-383e112c-cdfc-49ef-a4bb-a48ef2bdd0c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42c3e53f1af4451498a55a591762517b'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:04.812045 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1154/2334] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:22:04 2022] DELETE /identity/v3/users/42c3e53f1af4451498a55a591762517b => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:22:05.447333 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5894fc3-bd47-4536-9a69-0d5ab180e0af admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:05.449702 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b5894fc3-bd47-4536-9a69-0d5ab180e0af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:05.528535 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5894fc3-bd47-4536-9a69-0d5ab180e0af admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:05.654739 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:05.693979 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2843c1e3-9ae6-4fa7-832a-f79b16d8f36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:05.694792 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843c1e3-9ae6-4fa7-832a-f79b16d8f36d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:05.695318 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843c1e3-9ae6-4fa7-832a-f79b16d8f36d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:05.695837 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843c1e3-9ae6-4fa7-832a-f79b16d8f36d admin admin] PATH_INFO: `/v3/projects/1885171db7894a6497ded7c75e63ae52` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:05.697663 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2843c1e3-9ae6-4fa7-832a-f79b16d8f36d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1885171db7894a6497ded7c75e63ae52)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:05.705905 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2843c1e3-9ae6-4fa7-832a-f79b16d8f36d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:05.814204 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2843c1e3-9ae6-4fa7-832a-f79b16d8f36d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1885171db7894a6497ded7c75e63ae52', 'request_id': 'req-2843c1e3-9ae6-4fa7-832a-f79b16d8f36d'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:05.822561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2843c1e3-9ae6-4fa7-832a-f79b16d8f36d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1885171db7894a6497ded7c75e63ae52', 'request_id': 'req-2843c1e3-9ae6-4fa7-832a-f79b16d8f36d'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:05.832472 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2843c1e3-9ae6-4fa7-832a-f79b16d8f36d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1885171db7894a6497ded7c75e63ae52', 'request_id': 'req-2843c1e3-9ae6-4fa7-832a-f79b16d8f36d'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:05.852342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2843c1e3-9ae6-4fa7-832a-f79b16d8f36d admin admin] The token cache is being invalidate because project 1885171db7894a6497ded7c75e63ae52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:22:05.852669 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2843c1e3-9ae6-4fa7-832a-f79b16d8f36d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:05.853485 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2843c1e3-9ae6-4fa7-832a-f79b16d8f36d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:05.858475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1181/2335] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:22:05 2022] DELETE /identity/v3/projects/1885171db7894a6497ded7c75e63ae52 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:22:05.860231 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60777db4-f3e2-4bc7-95c6-0335122eb31d None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:05.861096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60777db4-f3e2-4bc7-95c6-0335122eb31d None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:05.861585 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60777db4-f3e2-4bc7-95c6-0335122eb31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:05.888574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-60777db4-f3e2-4bc7-95c6-0335122eb31d None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:05.890171 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-60777db4-f3e2-4bc7-95c6-0335122eb31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:05.990826 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1155/2336] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:22:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:05.998619 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea1f2f-0336-4fb1-9b96-f3d4ac080e7f None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:05.998851 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea1f2f-0336-4fb1-9b96-f3d4ac080e7f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:05.999126 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea1f2f-0336-4fb1-9b96-f3d4ac080e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:06.089246 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-0cea1f2f-0336-4fb1-9b96-f3d4ac080e7f None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:06.092632 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-0cea1f2f-0336-4fb1-9b96-f3d4ac080e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:06.121086 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1182/2337] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:22:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:06.122432 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60777db4-f3e2-4bc7-95c6-0335122eb31d None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:06.142231 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60777db4-f3e2-4bc7-95c6-0335122eb31d None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:06.149286 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60e709-6074-4018-a565-70092a1f19b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:06.150178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60e709-6074-4018-a565-70092a1f19b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:06.150497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60e709-6074-4018-a565-70092a1f19b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:06.150774 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60e709-6074-4018-a565-70092a1f19b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:06.152263 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b60e709-6074-4018-a565-70092a1f19b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:22:06.152610 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b60e709-6074-4018-a565-70092a1f19b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:06.154295 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b60e709-6074-4018-a565-70092a1f19b0 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:06.168812 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1156/2338] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:22:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:22:06.174189 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea1f2f-0336-4fb1-9b96-f3d4ac080e7f None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:06.186656 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea1f2f-0336-4fb1-9b96-f3d4ac080e7f None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:06.199461 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f0cf0-2a2e-4f7a-b14a-557f11ae10c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:06.200330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f0cf0-2a2e-4f7a-b14a-557f11ae10c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:06.200808 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f0cf0-2a2e-4f7a-b14a-557f11ae10c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:06.201307 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f0cf0-2a2e-4f7a-b14a-557f11ae10c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:06.202730 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f0cf0-2a2e-4f7a-b14a-557f11ae10c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:06.208276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f0cf0-2a2e-4f7a-b14a-557f11ae10c2 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:06.287828 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1183/2339] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:22:06 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:22:06.290022 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60e709-6074-4018-a565-70092a1f19b0 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:06.319004 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60e709-6074-4018-a565-70092a1f19b0 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:06.478727 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4eb5d0-c585-44b1-ba68-a6191ddd1514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:06.480691 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4eb5d0-c585-44b1-ba68-a6191ddd1514 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:06.481299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4eb5d0-c585-44b1-ba68-a6191ddd1514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:06.482073 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4eb5d0-c585-44b1-ba68-a6191ddd1514 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:06.483771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4eb5d0-c585-44b1-ba68-a6191ddd1514 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:06.491266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4eb5d0-c585-44b1-ba68-a6191ddd1514 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:06.592242 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1157/2340] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:22:06 2022] POST /identity/v3/users => generated 287 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:22:06.595181 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f0cf0-2a2e-4f7a-b14a-557f11ae10c2 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:06.627186 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f0cf0-2a2e-4f7a-b14a-557f11ae10c2 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:06.633738 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf3b36-e62f-4026-a244-af18a8fedcb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:06.634370 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf3b36-e62f-4026-a244-af18a8fedcb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:06.634651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf3b36-e62f-4026-a244-af18a8fedcb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:06.634926 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf3b36-e62f-4026-a244-af18a8fedcb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:06.635631 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf3b36-e62f-4026-a244-af18a8fedcb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:06.639035 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf3b36-e62f-4026-a244-af18a8fedcb6 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:06.656380 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1184/2341] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:22:06.658130 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4eb5d0-c585-44b1-ba68-a6191ddd1514 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:06.679761 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4eb5d0-c585-44b1-ba68-a6191ddd1514 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:06.687185 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92d666-1f21-4887-a9bc-4cf816d81591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:06.687894 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92d666-1f21-4887-a9bc-4cf816d81591 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:06.688599 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92d666-1f21-4887-a9bc-4cf816d81591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:06.688920 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92d666-1f21-4887-a9bc-4cf816d81591 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:06.689919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92d666-1f21-4887-a9bc-4cf816d81591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:06.692712 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92d666-1f21-4887-a9bc-4cf816d81591 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:06.724962 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1158/2342] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:22:06.726857 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf3b36-e62f-4026-a244-af18a8fedcb6 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:06.744846 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf3b36-e62f-4026-a244-af18a8fedcb6 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:06.760689 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd9494-3f64-44be-9632-12d8e7825ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:06.762040 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd9494-3f64-44be-9632-12d8e7825ac4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:06.762623 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd9494-3f64-44be-9632-12d8e7825ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:06.763255 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd9494-3f64-44be-9632-12d8e7825ac4 admin admin] PATH_INFO: `/v3/projects/85a04c77f22945659bab531c8db3c1cf/users/aecaa39cbcb54c639ed7168eb10b1d60/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:06.845823 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd9494-3f64-44be-9632-12d8e7825ac4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85a04c77f22945659bab531c8db3c1cf, user_id=aecaa39cbcb54c639ed7168eb10b1d60, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:06.857659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd9494-3f64-44be-9632-12d8e7825ac4 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:06.905924 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1185/2343] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:22:06 2022] PUT /identity/v3/projects/85a04c77f22945659bab531c8db3c1cf/users/aecaa39cbcb54c639ed7168eb10b1d60/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:22:06.918017 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14997479-cb85-4d3e-89d5-56b6115d50f7 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:06.918606 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14997479-cb85-4d3e-89d5-56b6115d50f7 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:06.918606 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14997479-cb85-4d3e-89d5-56b6115d50f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:06.963167 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-14997479-cb85-4d3e-89d5-56b6115d50f7 None None] MFA Rules not processed for user `aecaa39cbcb54c639ed7168eb10b1d60`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:06.964501 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-14997479-cb85-4d3e-89d5-56b6115d50f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:06.975728 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:07.242248 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1159/2344] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:22:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:07.259336 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd9494-3f64-44be-9632-12d8e7825ac4 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:07.261395 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-fcdd9494-3f64-44be-9632-12d8e7825ac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:07.345522 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd9494-3f64-44be-9632-12d8e7825ac4 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:07.351567 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e541cc-52e2-4e24-9c9e-3d085d322807 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:07.351567 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e541cc-52e2-4e24-9c9e-3d085d322807 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:07.352973 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e541cc-52e2-4e24-9c9e-3d085d322807 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:07.352973 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e541cc-52e2-4e24-9c9e-3d085d322807 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:07.378684 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e541cc-52e2-4e24-9c9e-3d085d322807 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:07.380303 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e541cc-52e2-4e24-9c9e-3d085d322807 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:07.414545 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1186/2345] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:22:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:22:07.534307 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14997479-cb85-4d3e-89d5-56b6115d50f7 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:07.678726 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14997479-cb85-4d3e-89d5-56b6115d50f7 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:07.684331 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8088cdf1-5890-4497-b910-0bf392c7fe80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:07.684331 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088cdf1-5890-4497-b910-0bf392c7fe80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:07.684331 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088cdf1-5890-4497-b910-0bf392c7fe80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:07.684936 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088cdf1-5890-4497-b910-0bf392c7fe80 admin admin] PATH_INFO: `/v3/users/aecaa39cbcb54c639ed7168eb10b1d60` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:07.686940 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8088cdf1-5890-4497-b910-0bf392c7fe80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aecaa39cbcb54c639ed7168eb10b1d60)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:07.689067 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8088cdf1-5890-4497-b910-0bf392c7fe80 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:07.792358 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8088cdf1-5890-4497-b910-0bf392c7fe80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aecaa39cbcb54c639ed7168eb10b1d60'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:07.805713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8088cdf1-5890-4497-b910-0bf392c7fe80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aecaa39cbcb54c639ed7168eb10b1d60'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:07.806598 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8088cdf1-5890-4497-b910-0bf392c7fe80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aecaa39cbcb54c639ed7168eb10b1d60'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:07.828735 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8088cdf1-5890-4497-b910-0bf392c7fe80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aecaa39cbcb54c639ed7168eb10b1d60'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:07.838875 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8088cdf1-5890-4497-b910-0bf392c7fe80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aecaa39cbcb54c639ed7168eb10b1d60'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:07.852093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b8108-fa0b-4456-9b07-3d5fe9046470 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:07.853172 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b8108-fa0b-4456-9b07-3d5fe9046470 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:07.853646 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b8108-fa0b-4456-9b07-3d5fe9046470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:07.857342 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1160/2346] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:22:07 2022] DELETE /identity/v3/users/aecaa39cbcb54c639ed7168eb10b1d60 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:22:07.884543 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:07.909566 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-932b8108-fa0b-4456-9b07-3d5fe9046470 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:07.916216 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-932b8108-fa0b-4456-9b07-3d5fe9046470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:08.081900 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1187/2347] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:08.094330 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8088cdf1-5890-4497-b910-0bf392c7fe80 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:08.097822 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-8088cdf1-5890-4497-b910-0bf392c7fe80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:08.194471 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:08.262504 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8088cdf1-5890-4497-b910-0bf392c7fe80 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:08.270255 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca396c36-e983-43d4-aedb-986747037352 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:08.271062 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca396c36-e983-43d4-aedb-986747037352 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:08.271352 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca396c36-e983-43d4-aedb-986747037352 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:08.271750 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca396c36-e983-43d4-aedb-986747037352 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:08.304825 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca396c36-e983-43d4-aedb-986747037352 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:08.306234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca396c36-e983-43d4-aedb-986747037352 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:08.336980 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1161/2348] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:22:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:22:08.977262 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b8108-fa0b-4456-9b07-3d5fe9046470 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:08.982816 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-932b8108-fa0b-4456-9b07-3d5fe9046470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:09.014707 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b8108-fa0b-4456-9b07-3d5fe9046470 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:09.027370 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a44e36-ecf4-42c9-bcc0-16303a3d07a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:09.028767 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a44e36-ecf4-42c9-bcc0-16303a3d07a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:09.029061 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a44e36-ecf4-42c9-bcc0-16303a3d07a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:09.029346 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a44e36-ecf4-42c9-bcc0-16303a3d07a2 admin admin] PATH_INFO: `/v3/projects/85a04c77f22945659bab531c8db3c1cf` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:09.034114 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a44e36-ecf4-42c9-bcc0-16303a3d07a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85a04c77f22945659bab531c8db3c1cf)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:09.049583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a44e36-ecf4-42c9-bcc0-16303a3d07a2 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:09.161375 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-19a44e36-ecf4-42c9-bcc0-16303a3d07a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85a04c77f22945659bab531c8db3c1cf', 'request_id': 'req-19a44e36-ecf4-42c9-bcc0-16303a3d07a2'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:09.177038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-19a44e36-ecf4-42c9-bcc0-16303a3d07a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85a04c77f22945659bab531c8db3c1cf', 'request_id': 'req-19a44e36-ecf4-42c9-bcc0-16303a3d07a2'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:09.187470 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-19a44e36-ecf4-42c9-bcc0-16303a3d07a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85a04c77f22945659bab531c8db3c1cf', 'request_id': 'req-19a44e36-ecf4-42c9-bcc0-16303a3d07a2'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:09.227262 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-19a44e36-ecf4-42c9-bcc0-16303a3d07a2 admin admin] The token cache is being invalidate because project 85a04c77f22945659bab531c8db3c1cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:22:09.229622 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-19a44e36-ecf4-42c9-bcc0-16303a3d07a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:09.230531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-19a44e36-ecf4-42c9-bcc0-16303a3d07a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:09.232423 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1188/2349] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:22:08 2022] DELETE /identity/v3/projects/85a04c77f22945659bab531c8db3c1cf => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:22:09.375100 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f376b2-232f-4548-8ae8-653ab59f8252 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:09.375946 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f376b2-232f-4548-8ae8-653ab59f8252 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:09.375946 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f376b2-232f-4548-8ae8-653ab59f8252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:09.483819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-15f376b2-232f-4548-8ae8-653ab59f8252 None None] MFA Rules not processed for user `8ff56403efde4509bc830967a1c25ccc`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:09.486219 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-15f376b2-232f-4548-8ae8-653ab59f8252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:09.620284 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1162/2350] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:22:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:09.625149 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a44e36-ecf4-42c9-bcc0-16303a3d07a2 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:09.626684 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-19a44e36-ecf4-42c9-bcc0-16303a3d07a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:09.707144 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a44e36-ecf4-42c9-bcc0-16303a3d07a2 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:09.712155 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cdb1b8-b0d8-42ce-b74c-e1bdd541b589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:09.712823 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cdb1b8-b0d8-42ce-b74c-e1bdd541b589 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:09.713103 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cdb1b8-b0d8-42ce-b74c-e1bdd541b589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:09.713376 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cdb1b8-b0d8-42ce-b74c-e1bdd541b589 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:09.714023 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cdb1b8-b0d8-42ce-b74c-e1bdd541b589 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:09.716023 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cdb1b8-b0d8-42ce-b74c-e1bdd541b589 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:09.882998 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1189/2351] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:22:09 2022] POST /identity/v3/projects => generated 373 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:22:09.888874 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f376b2-232f-4548-8ae8-653ab59f8252 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:09.902972 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f376b2-232f-4548-8ae8-653ab59f8252 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:10.015694 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ccc638-9ef1-46db-be95-b4e1d4657a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:10.016340 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ccc638-9ef1-46db-be95-b4e1d4657a92 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:10.016616 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ccc638-9ef1-46db-be95-b4e1d4657a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:10.016885 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ccc638-9ef1-46db-be95-b4e1d4657a92 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:10.017964 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ccc638-9ef1-46db-be95-b4e1d4657a92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:10.020831 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ccc638-9ef1-46db-be95-b4e1d4657a92 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:10.083058 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1163/2352] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:22:09 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:22:10.083296 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cdb1b8-b0d8-42ce-b74c-e1bdd541b589 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:10.116057 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cdb1b8-b0d8-42ce-b74c-e1bdd541b589 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:10.125150 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f7e79-3b1d-4100-8dca-552610e3ab60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:10.125795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f7e79-3b1d-4100-8dca-552610e3ab60 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:10.126066 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f7e79-3b1d-4100-8dca-552610e3ab60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:10.126330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f7e79-3b1d-4100-8dca-552610e3ab60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:10.127000 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f7e79-3b1d-4100-8dca-552610e3ab60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:10.128983 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f7e79-3b1d-4100-8dca-552610e3ab60 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:10.156257 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1190/2353] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:22:10.174151 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ccc638-9ef1-46db-be95-b4e1d4657a92 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:10.194041 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ccc638-9ef1-46db-be95-b4e1d4657a92 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:10.205495 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d8a8c-d0f8-4e79-90a0-f8102722cc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:10.207370 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d8a8c-d0f8-4e79-90a0-f8102722cc9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:10.207981 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d8a8c-d0f8-4e79-90a0-f8102722cc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:10.208761 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d8a8c-d0f8-4e79-90a0-f8102722cc9b admin admin] PATH_INFO: `/v3/projects/8a77c0c3fc9249ceb708c934a5c77df3/users/d9f4cca8aa5443c18c94bca796813c4f/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:10.234265 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d8a8c-d0f8-4e79-90a0-f8102722cc9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a77c0c3fc9249ceb708c934a5c77df3, user_id=d9f4cca8aa5443c18c94bca796813c4f, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:10.238652 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d8a8c-d0f8-4e79-90a0-f8102722cc9b admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:10.267582 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1164/2354] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:22:10 2022] PUT /identity/v3/projects/8a77c0c3fc9249ceb708c934a5c77df3/users/d9f4cca8aa5443c18c94bca796813c4f/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:22:10.282346 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f7e79-3b1d-4100-8dca-552610e3ab60 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:10.292669 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:10.300984 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f7e79-3b1d-4100-8dca-552610e3ab60 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:10.402956 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b8be7-8ff2-49ee-8d6e-02de6d66e759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:10.403637 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b8be7-8ff2-49ee-8d6e-02de6d66e759 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:10.403926 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b8be7-8ff2-49ee-8d6e-02de6d66e759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:10.404205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b8be7-8ff2-49ee-8d6e-02de6d66e759 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:10.405080 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6b8be7-8ff2-49ee-8d6e-02de6d66e759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:10.406982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6b8be7-8ff2-49ee-8d6e-02de6d66e759 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:10.424671 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1191/2355] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:22:10.425972 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d8a8c-d0f8-4e79-90a0-f8102722cc9b admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:10.437354 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d8a8c-d0f8-4e79-90a0-f8102722cc9b admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:10.442971 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8c9c6-b4a0-4c3d-8269-ba5a661c82e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:10.443839 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8c9c6-b4a0-4c3d-8269-ba5a661c82e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:10.444208 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8c9c6-b4a0-4c3d-8269-ba5a661c82e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:10.444556 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8c9c6-b4a0-4c3d-8269-ba5a661c82e7 admin admin] PATH_INFO: `/v3/projects/8a77c0c3fc9249ceb708c934a5c77df3/users/d9f4cca8aa5443c18c94bca796813c4f/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:10.446714 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad8c9c6-b4a0-4c3d-8269-ba5a661c82e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a77c0c3fc9249ceb708c934a5c77df3, user_id=d9f4cca8aa5443c18c94bca796813c4f, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:10.449677 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad8c9c6-b4a0-4c3d-8269-ba5a661c82e7 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:10.463504 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1165/2356] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:22:10 2022] PUT /identity/v3/projects/8a77c0c3fc9249ceb708c934a5c77df3/users/d9f4cca8aa5443c18c94bca796813c4f/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:22:17.710579 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188e343-e6cd-42c3-86f3-b3bade7dd53a None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:17.711378 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188e343-e6cd-42c3-86f3-b3bade7dd53a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:17.711378 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188e343-e6cd-42c3-86f3-b3bade7dd53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:17.743815 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-6188e343-e6cd-42c3-86f3-b3bade7dd53a None None] MFA Rules not processed for user `d9f4cca8aa5443c18c94bca796813c4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:17.745066 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6188e343-e6cd-42c3-86f3-b3bade7dd53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:17.869482 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1192/2357] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:22:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:17.892360 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8c9c6-b4a0-4c3d-8269-ba5a661c82e7 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:17.895279 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-1ad8c9c6-b4a0-4c3d-8269-ba5a661c82e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:17.915983 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:18.019996 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8c9c6-b4a0-4c3d-8269-ba5a661c82e7 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:18.029996 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674ab8a-3054-415e-beb0-31c8eca22bd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:18.031051 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674ab8a-3054-415e-beb0-31c8eca22bd8 service nova] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:18.031623 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674ab8a-3054-415e-beb0-31c8eca22bd8 service nova] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:18.032055 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674ab8a-3054-415e-beb0-31c8eca22bd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:18.137040 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674ab8a-3054-415e-beb0-31c8eca22bd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:18.138717 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674ab8a-3054-415e-beb0-31c8eca22bd8 service nova] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:18.152144 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1166/2358] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:22:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:22:20.707769 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daab5d1-2c5f-45dc-8c62-82be7c577556 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:20.708573 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daab5d1-2c5f-45dc-8c62-82be7c577556 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:20.708728 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daab5d1-2c5f-45dc-8c62-82be7c577556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:20.729141 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-4daab5d1-2c5f-45dc-8c62-82be7c577556 None None] MFA Rules not processed for user `8ff56403efde4509bc830967a1c25ccc`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:20.730894 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4daab5d1-2c5f-45dc-8c62-82be7c577556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:20.740909 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1193/2359] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:22:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:28.146355 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=50 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:30.723338 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=65 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:46.745870 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674ab8a-3054-415e-beb0-31c8eca22bd8 service nova] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:46.747290 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-1674ab8a-3054-415e-beb0-31c8eca22bd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:46.814748 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674ab8a-3054-415e-beb0-31c8eca22bd8 service nova] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:46.819389 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f10cc-6a3f-4b5d-a316-ae4ba309ffdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:46.820729 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f10cc-6a3f-4b5d-a316-ae4ba309ffdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:46.821000 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f10cc-6a3f-4b5d-a316-ae4ba309ffdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:46.821264 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f10cc-6a3f-4b5d-a316-ae4ba309ffdd admin admin] PATH_INFO: `/v3/users/5205ad3a6e4f46af92c2bff3a14255b4` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:46.822414 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29f10cc-6a3f-4b5d-a316-ae4ba309ffdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=5205ad3a6e4f46af92c2bff3a14255b4)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:46.824363 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29f10cc-6a3f-4b5d-a316-ae4ba309ffdd admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:46.881376 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b29f10cc-6a3f-4b5d-a316-ae4ba309ffdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5205ad3a6e4f46af92c2bff3a14255b4'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:46.890240 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b29f10cc-6a3f-4b5d-a316-ae4ba309ffdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5205ad3a6e4f46af92c2bff3a14255b4'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:46.890869 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b29f10cc-6a3f-4b5d-a316-ae4ba309ffdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5205ad3a6e4f46af92c2bff3a14255b4'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:46.908418 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b29f10cc-6a3f-4b5d-a316-ae4ba309ffdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5205ad3a6e4f46af92c2bff3a14255b4'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:46.919864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b29f10cc-6a3f-4b5d-a316-ae4ba309ffdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5205ad3a6e4f46af92c2bff3a14255b4'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:46.926992 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1167/2360] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:22:46 2022] DELETE /identity/v3/users/5205ad3a6e4f46af92c2bff3a14255b4 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:22:46.932897 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92782ad7-491d-4400-920f-7a51be0509f7 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:46.933858 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92782ad7-491d-4400-920f-7a51be0509f7 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:46.933858 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92782ad7-491d-4400-920f-7a51be0509f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:46.956618 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-92782ad7-491d-4400-920f-7a51be0509f7 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:46.957992 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-92782ad7-491d-4400-920f-7a51be0509f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:47.033339 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1194/2361] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:22:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:22:47.046960 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f10cc-6a3f-4b5d-a316-ae4ba309ffdd admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:47.052638 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-b29f10cc-6a3f-4b5d-a316-ae4ba309ffdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:47.127697 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f10cc-6a3f-4b5d-a316-ae4ba309ffdd admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:47.131768 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c46768-e943-4c3c-b1b8-d01e8b17a395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:47.132637 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c46768-e943-4c3c-b1b8-d01e8b17a395 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:47.132920 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c46768-e943-4c3c-b1b8-d01e8b17a395 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:47.133193 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c46768-e943-4c3c-b1b8-d01e8b17a395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:47.143155 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c46768-e943-4c3c-b1b8-d01e8b17a395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:47.144644 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c46768-e943-4c3c-b1b8-d01e8b17a395 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:47.155191 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1168/2362] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:22:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:22:47.439801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92782ad7-491d-4400-920f-7a51be0509f7 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:47.441291 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-92782ad7-491d-4400-920f-7a51be0509f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:47.453442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92782ad7-491d-4400-920f-7a51be0509f7 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:47.457963 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b5700f-2b65-455a-837b-b9f26a8bb0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:47.458595 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5700f-2b65-455a-837b-b9f26a8bb0c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:47.458875 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5700f-2b65-455a-837b-b9f26a8bb0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:47.459149 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5700f-2b65-455a-837b-b9f26a8bb0c3 admin admin] PATH_INFO: `/v3/projects/328457097fed40a6b9986d414e4fad7a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:47.460787 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b5700f-2b65-455a-837b-b9f26a8bb0c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=328457097fed40a6b9986d414e4fad7a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:47.462700 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b5700f-2b65-455a-837b-b9f26a8bb0c3 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:47.519010 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-44b5700f-2b65-455a-837b-b9f26a8bb0c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '328457097fed40a6b9986d414e4fad7a', 'request_id': 'req-44b5700f-2b65-455a-837b-b9f26a8bb0c3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:47.525412 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-44b5700f-2b65-455a-837b-b9f26a8bb0c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '328457097fed40a6b9986d414e4fad7a', 'request_id': 'req-44b5700f-2b65-455a-837b-b9f26a8bb0c3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:47.531884 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-44b5700f-2b65-455a-837b-b9f26a8bb0c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '328457097fed40a6b9986d414e4fad7a', 'request_id': 'req-44b5700f-2b65-455a-837b-b9f26a8bb0c3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:47.545096 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-44b5700f-2b65-455a-837b-b9f26a8bb0c3 admin admin] The token cache is being invalidate because project 328457097fed40a6b9986d414e4fad7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:22:47.545377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-44b5700f-2b65-455a-837b-b9f26a8bb0c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:47.546016 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-44b5700f-2b65-455a-837b-b9f26a8bb0c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:47.547485 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1195/2363] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:22:47 2022] DELETE /identity/v3/projects/328457097fed40a6b9986d414e4fad7a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:22:50.011768 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c46768-e943-4c3c-b1b8-d01e8b17a395 service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:50.013213 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-67c46768-e943-4c3c-b1b8-d01e8b17a395 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:50.026784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c46768-e943-4c3c-b1b8-d01e8b17a395 service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:50.045503 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:50.086473 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d122bb3-2da8-45a6-8a4d-93ff5a1c164e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:50.087096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d122bb3-2da8-45a6-8a4d-93ff5a1c164e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:50.087376 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d122bb3-2da8-45a6-8a4d-93ff5a1c164e admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:50.087687 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d122bb3-2da8-45a6-8a4d-93ff5a1c164e admin admin] PATH_INFO: `/v3/users/a6f376032e1a44c7865f564fabcc66da` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:50.088811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d122bb3-2da8-45a6-8a4d-93ff5a1c164e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6f376032e1a44c7865f564fabcc66da)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:50.090709 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d122bb3-2da8-45a6-8a4d-93ff5a1c164e admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:50.135819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8d122bb3-2da8-45a6-8a4d-93ff5a1c164e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6f376032e1a44c7865f564fabcc66da'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:50.143957 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8d122bb3-2da8-45a6-8a4d-93ff5a1c164e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6f376032e1a44c7865f564fabcc66da'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:50.144575 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8d122bb3-2da8-45a6-8a4d-93ff5a1c164e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6f376032e1a44c7865f564fabcc66da'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:50.158830 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8d122bb3-2da8-45a6-8a4d-93ff5a1c164e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6f376032e1a44c7865f564fabcc66da'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:50.166552 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8d122bb3-2da8-45a6-8a4d-93ff5a1c164e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6f376032e1a44c7865f564fabcc66da'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:50.173653 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1169/2364] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:22:50 2022] DELETE /identity/v3/users/a6f376032e1a44c7865f564fabcc66da => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:22:50.178286 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd4548-3850-44ed-9d57-12c3da92824a None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:50.178789 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd4548-3850-44ed-9d57-12c3da92824a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:50.178926 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd4548-3850-44ed-9d57-12c3da92824a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:50.198906 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-c9cd4548-3850-44ed-9d57-12c3da92824a None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:50.200177 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c9cd4548-3850-44ed-9d57-12c3da92824a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:50.261205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1196/2365] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:22:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:50.270990 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d122bb3-2da8-45a6-8a4d-93ff5a1c164e admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:50.272516 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-8d122bb3-2da8-45a6-8a4d-93ff5a1c164e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:50.334179 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d122bb3-2da8-45a6-8a4d-93ff5a1c164e admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:50.338504 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321168d3-5a24-4f46-af7a-43e5cbc6ec3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:50.339261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321168d3-5a24-4f46-af7a-43e5cbc6ec3a service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:50.339647 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321168d3-5a24-4f46-af7a-43e5cbc6ec3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:50.339740 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321168d3-5a24-4f46-af7a-43e5cbc6ec3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:50.349280 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321168d3-5a24-4f46-af7a-43e5cbc6ec3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:50.350747 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321168d3-5a24-4f46-af7a-43e5cbc6ec3a service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:50.360446 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1170/2366] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:22:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:22:50.764574 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd4548-3850-44ed-9d57-12c3da92824a None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:50.766752 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c9cd4548-3850-44ed-9d57-12c3da92824a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:50.780305 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd4548-3850-44ed-9d57-12c3da92824a None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:50.792315 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f09ef-625a-41ba-9e03-2a15cfa5c796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:50.793071 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f09ef-625a-41ba-9e03-2a15cfa5c796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:50.793454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f09ef-625a-41ba-9e03-2a15cfa5c796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:50.793454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f09ef-625a-41ba-9e03-2a15cfa5c796 admin admin] PATH_INFO: `/v3/projects/b8a8919343314702bbbb30f67a06d4db` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:50.794901 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41f09ef-625a-41ba-9e03-2a15cfa5c796 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8a8919343314702bbbb30f67a06d4db)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:50.796866 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41f09ef-625a-41ba-9e03-2a15cfa5c796 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:50.822286 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:50.852249 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d41f09ef-625a-41ba-9e03-2a15cfa5c796 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8a8919343314702bbbb30f67a06d4db', 'request_id': 'req-d41f09ef-625a-41ba-9e03-2a15cfa5c796'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:50.858007 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d41f09ef-625a-41ba-9e03-2a15cfa5c796 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8a8919343314702bbbb30f67a06d4db', 'request_id': 'req-d41f09ef-625a-41ba-9e03-2a15cfa5c796'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:50.864364 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d41f09ef-625a-41ba-9e03-2a15cfa5c796 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8a8919343314702bbbb30f67a06d4db', 'request_id': 'req-d41f09ef-625a-41ba-9e03-2a15cfa5c796'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:50.880365 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d41f09ef-625a-41ba-9e03-2a15cfa5c796 admin admin] The token cache is being invalidate because project b8a8919343314702bbbb30f67a06d4db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:22:50.880632 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d41f09ef-625a-41ba-9e03-2a15cfa5c796 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:50.881266 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d41f09ef-625a-41ba-9e03-2a15cfa5c796 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:22:50.886493 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1197/2367] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:22:50 2022] DELETE /identity/v3/projects/b8a8919343314702bbbb30f67a06d4db => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:22:50.889003 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbceec33-7d1e-45e2-a408-34f1e382e577 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:50.889311 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbceec33-7d1e-45e2-a408-34f1e382e577 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:50.889621 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbceec33-7d1e-45e2-a408-34f1e382e577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:50.911922 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-dbceec33-7d1e-45e2-a408-34f1e382e577 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:50.913793 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-dbceec33-7d1e-45e2-a408-34f1e382e577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:50.955671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:50.976807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1171/2368] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:22:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:22:50.984948 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5c3ea-375d-4023-a897-5342a695ff73 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:50.985255 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5c3ea-375d-4023-a897-5342a695ff73 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:50.985525 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5c3ea-375d-4023-a897-5342a695ff73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:51.005220 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-25a5c3ea-375d-4023-a897-5342a695ff73 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:51.006397 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-25a5c3ea-375d-4023-a897-5342a695ff73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:51.013995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1198/2369] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:22:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:51.018150 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbceec33-7d1e-45e2-a408-34f1e382e577 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:51.028337 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbceec33-7d1e-45e2-a408-34f1e382e577 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:51.032986 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9b61a-c76a-4266-90ff-cf1ec50f6675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:51.033580 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9b61a-c76a-4266-90ff-cf1ec50f6675 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:51.033856 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9b61a-c76a-4266-90ff-cf1ec50f6675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:51.034151 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9b61a-c76a-4266-90ff-cf1ec50f6675 admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:51.034789 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b9b61a-c76a-4266-90ff-cf1ec50f6675 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:22:51.035087 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b9b61a-c76a-4266-90ff-cf1ec50f6675 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:51.036758 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b9b61a-c76a-4266-90ff-cf1ec50f6675 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:51.049099 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1172/2370] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:22:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:22:51.055061 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a5c3ea-375d-4023-a897-5342a695ff73 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:51.063798 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a5c3ea-375d-4023-a897-5342a695ff73 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:51.068455 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74776699-2726-4481-af4a-e922d5479a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:51.069046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74776699-2726-4481-af4a-e922d5479a72 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:51.069324 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74776699-2726-4481-af4a-e922d5479a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:51.069595 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74776699-2726-4481-af4a-e922d5479a72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:51.070231 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74776699-2726-4481-af4a-e922d5479a72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:51.072151 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74776699-2726-4481-af4a-e922d5479a72 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:51.099356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1199/2371] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:22:51 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:22:51.104044 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9b61a-c76a-4266-90ff-cf1ec50f6675 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:51.113915 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9b61a-c76a-4266-90ff-cf1ec50f6675 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:51.172033 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74578b0-7dad-4c42-acea-5a8fb8afe38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:51.172627 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74578b0-7dad-4c42-acea-5a8fb8afe38b admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:51.172901 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74578b0-7dad-4c42-acea-5a8fb8afe38b admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:51.173168 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74578b0-7dad-4c42-acea-5a8fb8afe38b admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:51.173856 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74578b0-7dad-4c42-acea-5a8fb8afe38b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:51.175777 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74578b0-7dad-4c42-acea-5a8fb8afe38b admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:51.199034 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1173/2372] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:22:51 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:22:51.203539 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74776699-2726-4481-af4a-e922d5479a72 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:51.214367 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74776699-2726-4481-af4a-e922d5479a72 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:51.219077 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16663b0-f7cc-4888-a60f-e95777448d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:51.219704 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16663b0-f7cc-4888-a60f-e95777448d80 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:51.219987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16663b0-f7cc-4888-a60f-e95777448d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:51.220309 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16663b0-f7cc-4888-a60f-e95777448d80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:51.220885 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16663b0-f7cc-4888-a60f-e95777448d80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:51.222724 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16663b0-f7cc-4888-a60f-e95777448d80 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:51.234995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1200/2373] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:22:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:22:51.238903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74578b0-7dad-4c42-acea-5a8fb8afe38b admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:51.251701 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74578b0-7dad-4c42-acea-5a8fb8afe38b admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:51.257411 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5a462-7cec-49a1-8699-7b5b2eafc4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:51.258133 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5a462-7cec-49a1-8699-7b5b2eafc4ba admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:51.258488 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5a462-7cec-49a1-8699-7b5b2eafc4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:51.258837 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5a462-7cec-49a1-8699-7b5b2eafc4ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:51.259599 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5a462-7cec-49a1-8699-7b5b2eafc4ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:51.261508 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5a462-7cec-49a1-8699-7b5b2eafc4ba admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:51.277438 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1174/2374] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:22:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:22:51.281900 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16663b0-f7cc-4888-a60f-e95777448d80 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:51.293074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16663b0-f7cc-4888-a60f-e95777448d80 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:51.300437 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c839d9b-89c9-4c40-bf8c-4219c205b0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:51.301233 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c839d9b-89c9-4c40-bf8c-4219c205b0e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:51.301517 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c839d9b-89c9-4c40-bf8c-4219c205b0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:51.301789 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c839d9b-89c9-4c40-bf8c-4219c205b0e3 admin admin] PATH_INFO: `/v3/projects/5d891c413f984a409b379359a992000a/users/ab0854279bf244198ed61ddff0279655/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:51.320426 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c839d9b-89c9-4c40-bf8c-4219c205b0e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d891c413f984a409b379359a992000a, user_id=ab0854279bf244198ed61ddff0279655, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:51.323084 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c839d9b-89c9-4c40-bf8c-4219c205b0e3 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:51.337462 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1201/2375] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:22:51 2022] PUT /identity/v3/projects/5d891c413f984a409b379359a992000a/users/ab0854279bf244198ed61ddff0279655/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:22:51.347673 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9ee29-4632-40a7-9850-ec769af2e23d None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:51.347966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9ee29-4632-40a7-9850-ec769af2e23d None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:51.348285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9ee29-4632-40a7-9850-ec769af2e23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:51.377544 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-22a9ee29-4632-40a7-9850-ec769af2e23d None None] MFA Rules not processed for user `ab0854279bf244198ed61ddff0279655`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:51.378767 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-22a9ee29-4632-40a7-9850-ec769af2e23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:51.445022 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1175/2376] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:22:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:51.455795 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c839d9b-89c9-4c40-bf8c-4219c205b0e3 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:51.457525 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1c839d9b-89c9-4c40-bf8c-4219c205b0e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:51.512901 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c839d9b-89c9-4c40-bf8c-4219c205b0e3 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:51.516770 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b8556-eb24-46c9-9d84-e769dce35b88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:51.517424 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b8556-eb24-46c9-9d84-e769dce35b88 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:51.517702 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b8556-eb24-46c9-9d84-e769dce35b88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:51.517979 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b8556-eb24-46c9-9d84-e769dce35b88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:51.527358 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b8556-eb24-46c9-9d84-e769dce35b88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:51.529948 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b8556-eb24-46c9-9d84-e769dce35b88 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:51.545578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1202/2377] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:22:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:22:54.052944 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc26dfa-21f0-4c4b-9f26-a98116d82638 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:54.053474 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc26dfa-21f0-4c4b-9f26-a98116d82638 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:54.053474 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc26dfa-21f0-4c4b-9f26-a98116d82638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:54.072917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-2cc26dfa-21f0-4c4b-9f26-a98116d82638 None None] MFA Rules not processed for user `ab0854279bf244198ed61ddff0279655`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:54.074168 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-2cc26dfa-21f0-4c4b-9f26-a98116d82638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:54.081398 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1176/2378] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:22:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:22:56.066786 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a6e0c-1075-4126-a716-f4b387ee080b None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:56.067403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a6e0c-1075-4126-a716-f4b387ee080b None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:56.067403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a6e0c-1075-4126-a716-f4b387ee080b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:56.090663 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-078a6e0c-1075-4126-a716-f4b387ee080b None None] MFA Rules not processed for user `ff0999bd21bf4db5981053922200be2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:56.092089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-078a6e0c-1075-4126-a716-f4b387ee080b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:56.140944 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:56.159388 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1203/2379] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:22:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:23:04.067754 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=82,keystone:INSERT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:06.154612 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:39.023856 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528a6c0-f8ca-41ec-b3d6-33eae47330e3 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:39.024728 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528a6c0-f8ca-41ec-b3d6-33eae47330e3 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:39.024728 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528a6c0-f8ca-41ec-b3d6-33eae47330e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:39.044989 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-0528a6c0-f8ca-41ec-b3d6-33eae47330e3 None None] MFA Rules not processed for user `8c5ae98336c441118d01f46a50f732d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:23:39.046227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-0528a6c0-f8ca-41ec-b3d6-33eae47330e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:39.128652 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1177/2380] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Sep 5 12:23:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:23:39.180741 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a6e0c-1075-4126-a716-f4b387ee080b None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:23:39.182407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-078a6e0c-1075-4126-a716-f4b387ee080b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:39.266269 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a6e0c-1075-4126-a716-f4b387ee080b None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:23:39.273330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed322834-ca78-4708-bf8b-b4f41dae84b1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '857eb7657e7d45938baca17f91a16901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:23:39.274100 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed322834-ca78-4708-bf8b-b4f41dae84b1 service swift] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:39.274574 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed322834-ca78-4708-bf8b-b4f41dae84b1 service swift] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:39.275041 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed322834-ca78-4708-bf8b-b4f41dae84b1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:39.293102 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed322834-ca78-4708-bf8b-b4f41dae84b1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:23:39.294700 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed322834-ca78-4708-bf8b-b4f41dae84b1 service swift] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:23:39.307369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1204/2381] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Sep 5 12:23:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 12:23:39.894730 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f0ae5-aff5-46ae-a567-89b66ef583d9 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:39.894730 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f0ae5-aff5-46ae-a567-89b66ef583d9 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:39.894730 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f0ae5-aff5-46ae-a567-89b66ef583d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:39.920047 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-ba0f0ae5-aff5-46ae-a567-89b66ef583d9 None None] MFA Rules not processed for user `8c5ae98336c441118d01f46a50f732d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:23:39.921293 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-ba0f0ae5-aff5-46ae-a567-89b66ef583d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:39.930409 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1178/2382] 10.222.0.16 () {58 vars in 967 bytes} [Mon Sep 5 12:23:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:23:39.951123 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed322834-ca78-4708-bf8b-b4f41dae84b1 service swift] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:23:39.970175 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed322834-ca78-4708-bf8b-b4f41dae84b1 service swift] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:23:39.975464 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15588d90-2b83-4683-832c-1111b46ac307 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '857eb7657e7d45938baca17f91a16901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:23:39.976603 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15588d90-2b83-4683-832c-1111b46ac307 service swift] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:39.976877 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15588d90-2b83-4683-832c-1111b46ac307 service swift] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:39.977146 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15588d90-2b83-4683-832c-1111b46ac307 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:39.987195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15588d90-2b83-4683-832c-1111b46ac307 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:23:39.989052 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15588d90-2b83-4683-832c-1111b46ac307 service swift] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:23:39.998776 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1205/2383] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Sep 5 12:23:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 12:23:40.241760 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab485332-2068-4bbb-9399-382b59f9bd9d None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:40.242220 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab485332-2068-4bbb-9399-382b59f9bd9d None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:40.242335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab485332-2068-4bbb-9399-382b59f9bd9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:40.261616 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-ab485332-2068-4bbb-9399-382b59f9bd9d None None] MFA Rules not processed for user `8c5ae98336c441118d01f46a50f732d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:23:40.262810 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-ab485332-2068-4bbb-9399-382b59f9bd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:40.270074 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1179/2384] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Sep 5 12:23:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:23:40.282530 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15588d90-2b83-4683-832c-1111b46ac307 service swift] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:23:40.293368 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15588d90-2b83-4683-832c-1111b46ac307 service swift] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:23:40.298233 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87586e6-952e-4347-88e9-6e6bac750a8a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '857eb7657e7d45938baca17f91a16901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:23:40.298812 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87586e6-952e-4347-88e9-6e6bac750a8a service swift] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:40.299087 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87586e6-952e-4347-88e9-6e6bac750a8a service swift] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:40.299360 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87586e6-952e-4347-88e9-6e6bac750a8a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:40.307746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87586e6-952e-4347-88e9-6e6bac750a8a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:23:40.309332 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87586e6-952e-4347-88e9-6e6bac750a8a service swift] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:23:40.317161 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1206/2385] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Sep 5 12:23:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 12:23:40.661010 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c3c7c-6030-4257-bd0a-1dc1cba7ea65 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:40.661304 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c3c7c-6030-4257-bd0a-1dc1cba7ea65 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:40.661621 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c3c7c-6030-4257-bd0a-1dc1cba7ea65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:40.682570 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-ae4c3c7c-6030-4257-bd0a-1dc1cba7ea65 None None] MFA Rules not processed for user `8c5ae98336c441118d01f46a50f732d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:23:40.683835 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-ae4c3c7c-6030-4257-bd0a-1dc1cba7ea65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:40.691079 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1180/2386] 10.222.0.16 () {58 vars in 967 bytes} [Mon Sep 5 12:23:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:23:40.704318 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87586e6-952e-4347-88e9-6e6bac750a8a service swift] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:23:40.716636 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87586e6-952e-4347-88e9-6e6bac750a8a service swift] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:23:40.721733 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aec2f5-bcd8-46f9-81f5-a09bee6cef0f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '857eb7657e7d45938baca17f91a16901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:23:40.722373 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aec2f5-bcd8-46f9-81f5-a09bee6cef0f service swift] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:40.722649 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aec2f5-bcd8-46f9-81f5-a09bee6cef0f service swift] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:40.722920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aec2f5-bcd8-46f9-81f5-a09bee6cef0f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:40.731775 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aec2f5-bcd8-46f9-81f5-a09bee6cef0f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:23:40.733634 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aec2f5-bcd8-46f9-81f5-a09bee6cef0f service swift] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:23:40.747051 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1207/2387] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Sep 5 12:23:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 12:23:43.434032 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c3c7c-6030-4257-bd0a-1dc1cba7ea65 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:23:43.435554 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-ae4c3c7c-6030-4257-bd0a-1dc1cba7ea65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:43.503057 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c3c7c-6030-4257-bd0a-1dc1cba7ea65 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:23:43.507290 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd960060-72a7-4cba-9d9c-bc6355301c7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:23:43.508061 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd960060-72a7-4cba-9d9c-bc6355301c7c service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:43.508214 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd960060-72a7-4cba-9d9c-bc6355301c7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:43.508489 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd960060-72a7-4cba-9d9c-bc6355301c7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:43.510451 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-bd960060-72a7-4cba-9d9c-bc6355301c7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:43.591211 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd960060-72a7-4cba-9d9c-bc6355301c7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:23:43.592926 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd960060-72a7-4cba-9d9c-bc6355301c7c service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:23:43.605013 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1181/2388] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:23:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:23:47.177923 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aec2f5-bcd8-46f9-81f5-a09bee6cef0f service swift] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:23:47.267150 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aec2f5-bcd8-46f9-81f5-a09bee6cef0f service swift] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:23:47.273029 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd97ea8-0e56-4d13-9f06-407d4aebafe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:23:47.273768 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd97ea8-0e56-4d13-9f06-407d4aebafe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:47.276039 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd97ea8-0e56-4d13-9f06-407d4aebafe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:47.276335 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd97ea8-0e56-4d13-9f06-407d4aebafe1 admin admin] PATH_INFO: `/v3/users/ab0854279bf244198ed61ddff0279655` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:47.277562 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd97ea8-0e56-4d13-9f06-407d4aebafe1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab0854279bf244198ed61ddff0279655)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:23:47.279486 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd97ea8-0e56-4d13-9f06-407d4aebafe1 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:23:47.335302 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6bd97ea8-0e56-4d13-9f06-407d4aebafe1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab0854279bf244198ed61ddff0279655'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:23:47.341128 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:47.344674 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6bd97ea8-0e56-4d13-9f06-407d4aebafe1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab0854279bf244198ed61ddff0279655'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:23:47.345373 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6bd97ea8-0e56-4d13-9f06-407d4aebafe1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab0854279bf244198ed61ddff0279655'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:23:47.345964 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6bd97ea8-0e56-4d13-9f06-407d4aebafe1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab0854279bf244198ed61ddff0279655'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:23:47.359793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6bd97ea8-0e56-4d13-9f06-407d4aebafe1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab0854279bf244198ed61ddff0279655'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:23:47.372407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1208/2389] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:23:47 2022] DELETE /identity/v3/users/ab0854279bf244198ed61ddff0279655 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:23:47.377193 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcc0f0-fa49-4c85-adb2-847a825e81b2 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:47.377647 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcc0f0-fa49-4c85-adb2-847a825e81b2 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:47.377749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcc0f0-fa49-4c85-adb2-847a825e81b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:47.401802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-37dcc0f0-fa49-4c85-adb2-847a825e81b2 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:23:47.402986 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-37dcc0f0-fa49-4c85-adb2-847a825e81b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:47.426227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:47.471589 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1182/2390] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:23:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:23:47.482893 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd97ea8-0e56-4d13-9f06-407d4aebafe1 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:23:47.484925 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6bd97ea8-0e56-4d13-9f06-407d4aebafe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:47.550532 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd97ea8-0e56-4d13-9f06-407d4aebafe1 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:23:47.554953 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97bfd85-3d05-499e-b403-5891b7fa312a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:23:47.555636 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97bfd85-3d05-499e-b403-5891b7fa312a service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:47.555915 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97bfd85-3d05-499e-b403-5891b7fa312a service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:47.556237 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97bfd85-3d05-499e-b403-5891b7fa312a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:47.565751 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97bfd85-3d05-499e-b403-5891b7fa312a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:23:47.567105 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97bfd85-3d05-499e-b403-5891b7fa312a service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:23:47.577498 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1209/2391] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:23:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:23:47.753178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dcc0f0-fa49-4c85-adb2-847a825e81b2 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:23:47.754620 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-37dcc0f0-fa49-4c85-adb2-847a825e81b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:47.769119 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dcc0f0-fa49-4c85-adb2-847a825e81b2 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:23:47.774220 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65060e79-113f-4cb2-abd9-219078e71f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:23:47.775270 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65060e79-113f-4cb2-abd9-219078e71f66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:47.775893 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65060e79-113f-4cb2-abd9-219078e71f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:47.776180 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65060e79-113f-4cb2-abd9-219078e71f66 admin admin] PATH_INFO: `/v3/projects/5d891c413f984a409b379359a992000a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:47.777581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65060e79-113f-4cb2-abd9-219078e71f66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d891c413f984a409b379359a992000a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:23:47.779539 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65060e79-113f-4cb2-abd9-219078e71f66 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:23:47.834370 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-65060e79-113f-4cb2-abd9-219078e71f66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d891c413f984a409b379359a992000a', 'request_id': 'req-65060e79-113f-4cb2-abd9-219078e71f66'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:23:47.841791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-65060e79-113f-4cb2-abd9-219078e71f66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d891c413f984a409b379359a992000a', 'request_id': 'req-65060e79-113f-4cb2-abd9-219078e71f66'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:23:47.847290 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-65060e79-113f-4cb2-abd9-219078e71f66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d891c413f984a409b379359a992000a', 'request_id': 'req-65060e79-113f-4cb2-abd9-219078e71f66'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:23:47.858853 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-65060e79-113f-4cb2-abd9-219078e71f66 admin admin] The token cache is being invalidate because project 5d891c413f984a409b379359a992000a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:23:47.859135 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-65060e79-113f-4cb2-abd9-219078e71f66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:23:47.859799 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-65060e79-113f-4cb2-abd9-219078e71f66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:23:47.861286 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1183/2392] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:23:47 2022] DELETE /identity/v3/projects/5d891c413f984a409b379359a992000a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:23:48.222388 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97bfd85-3d05-499e-b403-5891b7fa312a service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:23:48.223913 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e97bfd85-3d05-499e-b403-5891b7fa312a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:48.239660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97bfd85-3d05-499e-b403-5891b7fa312a service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:23:48.303666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac70a3-2a1d-41e2-9917-627041e89550 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:23:48.304446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac70a3-2a1d-41e2-9917-627041e89550 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:48.304730 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac70a3-2a1d-41e2-9917-627041e89550 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:48.305008 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac70a3-2a1d-41e2-9917-627041e89550 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:48.307094 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-caac70a3-2a1d-41e2-9917-627041e89550 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:48.401828 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caac70a3-2a1d-41e2-9917-627041e89550 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:23:48.403385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caac70a3-2a1d-41e2-9917-627041e89550 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:23:48.415446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1210/2393] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:23:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:23:53.107300 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65060e79-113f-4cb2-abd9-219078e71f66 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:23:53.108807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-65060e79-113f-4cb2-abd9-219078e71f66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:53.127126 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65060e79-113f-4cb2-abd9-219078e71f66 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:23:53.133031 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade86d3d-a78e-4b38-b68f-0263bc61940b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:23:53.133716 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade86d3d-a78e-4b38-b68f-0263bc61940b service nova] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:53.134066 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade86d3d-a78e-4b38-b68f-0263bc61940b service nova] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:53.134385 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade86d3d-a78e-4b38-b68f-0263bc61940b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:53.136259 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-ade86d3d-a78e-4b38-b68f-0263bc61940b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:53.198911 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade86d3d-a78e-4b38-b68f-0263bc61940b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:23:53.200435 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade86d3d-a78e-4b38-b68f-0263bc61940b service nova] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:23:53.209750 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1184/2394] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:23:53 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:23:55.594237 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8dc253-f928-4f2a-9cb3-8d4c1b8adf7e None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:55.594748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8dc253-f928-4f2a-9cb3-8d4c1b8adf7e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:55.594911 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8dc253-f928-4f2a-9cb3-8d4c1b8adf7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:55.623479 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-db8dc253-f928-4f2a-9cb3-8d4c1b8adf7e None None] MFA Rules not processed for user `4e187162f3e64acdb8cd24f660ceeca6`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:23:55.625679 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-db8dc253-f928-4f2a-9cb3-8d4c1b8adf7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:55.634692 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:55.698388 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1211/2395] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:23:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:23:59.017456 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a12f48-fe8a-4c16-81a8-f50f1eb6463a None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:59.018285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a12f48-fe8a-4c16-81a8-f50f1eb6463a None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:59.018741 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a12f48-fe8a-4c16-81a8-f50f1eb6463a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:59.040385 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-78a12f48-fe8a-4c16-81a8-f50f1eb6463a None None] MFA Rules not processed for user `8c5ae98336c441118d01f46a50f732d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:23:59.042030 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-78a12f48-fe8a-4c16-81a8-f50f1eb6463a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:59.101114 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:59.125018 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1185/2396] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Sep 5 12:23:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:23:59.141205 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8dc253-f928-4f2a-9cb3-8d4c1b8adf7e None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:23:59.142648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-db8dc253-f928-4f2a-9cb3-8d4c1b8adf7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:59.218752 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8dc253-f928-4f2a-9cb3-8d4c1b8adf7e None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:23:59.223815 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec110b3-727f-4ccf-b047-255f73266d9d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '857eb7657e7d45938baca17f91a16901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:23:59.224422 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec110b3-727f-4ccf-b047-255f73266d9d service swift] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:59.224701 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec110b3-727f-4ccf-b047-255f73266d9d service swift] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:59.224975 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec110b3-727f-4ccf-b047-255f73266d9d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:59.234510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec110b3-727f-4ccf-b047-255f73266d9d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:23:59.236168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec110b3-727f-4ccf-b047-255f73266d9d service swift] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:23:59.246387 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1212/2397] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Sep 5 12:23:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 12:23:59.385335 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a12f48-fe8a-4c16-81a8-f50f1eb6463a None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:23:59.387130 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-78a12f48-fe8a-4c16-81a8-f50f1eb6463a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:59.410098 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a12f48-fe8a-4c16-81a8-f50f1eb6463a None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:23:59.415466 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2ac3e-f17c-4d22-a868-610a91cdf542 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:23:59.416425 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2ac3e-f17c-4d22-a868-610a91cdf542 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:59.416916 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2ac3e-f17c-4d22-a868-610a91cdf542 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:59.417373 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2ac3e-f17c-4d22-a868-610a91cdf542 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:59.419406 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-8ed2ac3e-f17c-4d22-a868-610a91cdf542 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:59.626783 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed2ac3e-f17c-4d22-a868-610a91cdf542 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:23:59.628535 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed2ac3e-f17c-4d22-a868-610a91cdf542 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:23:59.653682 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1186/2398] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:23:59 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:24:09.241934 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=51 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:24:09.647085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=33 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:24:14.050302 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec110b3-727f-4ccf-b047-255f73266d9d service swift] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:24:14.062302 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec110b3-727f-4ccf-b047-255f73266d9d service swift] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:24:14.066869 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652879a0-c39d-40d5-9b17-023f43e697d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:24:14.067633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652879a0-c39d-40d5-9b17-023f43e697d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:24:14.067982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652879a0-c39d-40d5-9b17-023f43e697d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:24:14.068371 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652879a0-c39d-40d5-9b17-023f43e697d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:24:14.070486 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-652879a0-c39d-40d5-9b17-023f43e697d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:24:14.081741 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652879a0-c39d-40d5-9b17-023f43e697d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:24:14.083343 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652879a0-c39d-40d5-9b17-023f43e697d7 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:24:14.093188 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1213/2399] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:24:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:24:19.115368 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2ac3e-f17c-4d22-a868-610a91cdf542 service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:24:19.126362 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2ac3e-f17c-4d22-a868-610a91cdf542 service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:24:19.134977 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f47b24-3a49-4259-9968-083b0eb9ea0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:24:19.135503 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f47b24-3a49-4259-9968-083b0eb9ea0a service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:24:19.135813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f47b24-3a49-4259-9968-083b0eb9ea0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:24:19.136081 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f47b24-3a49-4259-9968-083b0eb9ea0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:24:19.137850 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-26f47b24-3a49-4259-9968-083b0eb9ea0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:24:19.149491 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f47b24-3a49-4259-9968-083b0eb9ea0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:24:19.151545 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f47b24-3a49-4259-9968-083b0eb9ea0a service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:24:19.160590 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1187/2400] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:24:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:24:19.297301 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b9fe4-3fac-4787-a76d-7502264b088a None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:24:19.298090 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b9fe4-3fac-4787-a76d-7502264b088a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:24:19.298090 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b9fe4-3fac-4787-a76d-7502264b088a None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:24:19.299456 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1214/2401] 10.222.0.16 () {54 vars in 921 bytes} [Mon Sep 5 12:24:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:24:19.307141 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3cd40-28b6-40d4-bfdf-ec205fdc29bc None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:24:19.307729 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3cd40-28b6-40d4-bfdf-ec205fdc29bc None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:24:19.307906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3cd40-28b6-40d4-bfdf-ec205fdc29bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:24:19.331922 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-e5f3cd40-28b6-40d4-bfdf-ec205fdc29bc None None] MFA Rules not processed for user `7cda290418504f0db85d665479a05353`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:24:19.333416 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-e5f3cd40-28b6-40d4-bfdf-ec205fdc29bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:24:19.347368 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1188/2402] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Sep 5 12:24:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:24:19.360226 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b9fe4-3fac-4787-a76d-7502264b088a None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:24:19.362196 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5d0b9fe4-3fac-4787-a76d-7502264b088a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:24:19.459082 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b9fe4-3fac-4787-a76d-7502264b088a None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:24:19.465872 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31609d9-8517-4a64-ad3e-8b40ac8560e1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed75c0be809d4a0eb0e2ccc59805b1ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:24:19.466901 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31609d9-8517-4a64-ad3e-8b40ac8560e1 service glance] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:24:19.467402 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31609d9-8517-4a64-ad3e-8b40ac8560e1 service glance] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:24:19.467922 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31609d9-8517-4a64-ad3e-8b40ac8560e1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:24:19.487003 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31609d9-8517-4a64-ad3e-8b40ac8560e1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:24:19.489693 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31609d9-8517-4a64-ad3e-8b40ac8560e1 service glance] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:24:19.507118 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1215/2403] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:24:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:24:19.604059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc062b6d-e273-44a6-bdc2-fb846c7807ff None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:24:19.604356 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc062b6d-e273-44a6-bdc2-fb846c7807ff None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:24:19.604618 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc062b6d-e273-44a6-bdc2-fb846c7807ff None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:24:19.605981 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1189/2404] 10.222.0.16 () {54 vars in 921 bytes} [Mon Sep 5 12:24:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:24:19.611633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe1911-f879-4c52-9dea-0f55754daffa None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:24:19.611919 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe1911-f879-4c52-9dea-0f55754daffa None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:24:19.612201 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe1911-f879-4c52-9dea-0f55754daffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:24:19.634091 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-23fe1911-f879-4c52-9dea-0f55754daffa None None] MFA Rules not processed for user `9f79599ed0b74bdbab198627ef72252a`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:24:19.635317 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-23fe1911-f879-4c52-9dea-0f55754daffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:24:19.642189 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1216/2405] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Sep 5 12:24:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:24:19.655653 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc062b6d-e273-44a6-bdc2-fb846c7807ff None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:24:19.657712 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-bc062b6d-e273-44a6-bdc2-fb846c7807ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:24:19.671694 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc062b6d-e273-44a6-bdc2-fb846c7807ff None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:24:19.675957 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb4a54-40bb-4b8b-924d-8e8961d672c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:24:19.676615 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb4a54-40bb-4b8b-924d-8e8961d672c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:24:19.677001 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb4a54-40bb-4b8b-924d-8e8961d672c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:24:19.677319 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb4a54-40bb-4b8b-924d-8e8961d672c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:24:19.685313 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb4a54-40bb-4b8b-924d-8e8961d672c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:24:19.686734 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb4a54-40bb-4b8b-924d-8e8961d672c7 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:24:19.695795 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1190/2406] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:24:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:24:19.784353 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c196e-fb10-4d5d-aef2-4374658dd49e None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:24:19.784816 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c196e-fb10-4d5d-aef2-4374658dd49e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:24:19.785154 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c196e-fb10-4d5d-aef2-4374658dd49e None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:24:19.787182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1217/2407] 10.222.0.16 () {54 vars in 921 bytes} [Mon Sep 5 12:24:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:24:19.792729 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4796d79-25cd-49f2-b803-b292049554a2 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:24:19.793023 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4796d79-25cd-49f2-b803-b292049554a2 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:24:19.793302 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4796d79-25cd-49f2-b803-b292049554a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:24:19.814537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-b4796d79-25cd-49f2-b803-b292049554a2 None None] MFA Rules not processed for user `7cda290418504f0db85d665479a05353`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:24:19.815898 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-b4796d79-25cd-49f2-b803-b292049554a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:24:19.825973 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1191/2408] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Sep 5 12:24:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:24:19.838880 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c196e-fb10-4d5d-aef2-4374658dd49e None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:24:19.851648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c196e-fb10-4d5d-aef2-4374658dd49e None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:24:19.855600 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406e930-7ba3-46b3-bccf-0ba910f3cab5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:24:19.856230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406e930-7ba3-46b3-bccf-0ba910f3cab5 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:24:19.856507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406e930-7ba3-46b3-bccf-0ba910f3cab5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:24:19.857188 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406e930-7ba3-46b3-bccf-0ba910f3cab5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:24:19.869297 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4406e930-7ba3-46b3-bccf-0ba910f3cab5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:24:19.870964 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4406e930-7ba3-46b3-bccf-0ba910f3cab5 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:24:19.884970 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1218/2409] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:24:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:24:19.952449 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4796d79-25cd-49f2-b803-b292049554a2 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:24:19.953848 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-b4796d79-25cd-49f2-b803-b292049554a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:24:20.042114 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4796d79-25cd-49f2-b803-b292049554a2 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:24:20.048071 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc85546-0e0b-4693-857d-2b687d759f31 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524bcfffcdfa4790885e0d4cbd66fa80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:24:20.048869 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc85546-0e0b-4693-857d-2b687d759f31 service placement] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:24:20.049148 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc85546-0e0b-4693-857d-2b687d759f31 service placement] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:24:20.049451 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc85546-0e0b-4693-857d-2b687d759f31 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:24:20.051488 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-1fc85546-0e0b-4693-857d-2b687d759f31 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:24:20.064544 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc85546-0e0b-4693-857d-2b687d759f31 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:24:20.066304 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc85546-0e0b-4693-857d-2b687d759f31 service placement] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:24:20.078754 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1192/2410] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:24:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:24:29.877867 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=41 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:24:30.074344 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=50 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:24:30.548779 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12267749-9423-47bb-90d6-9ee943236f77 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:24:30.549144 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12267749-9423-47bb-90d6-9ee943236f77 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:24:30.549495 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12267749-9423-47bb-90d6-9ee943236f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:24:30.569586 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-12267749-9423-47bb-90d6-9ee943236f77 None None] MFA Rules not processed for user `8c5ae98336c441118d01f46a50f732d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:24:30.570794 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-12267749-9423-47bb-90d6-9ee943236f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:24:30.578084 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1219/2411] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Sep 5 12:24:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:24:30.591067 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc85546-0e0b-4693-857d-2b687d759f31 service placement] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:24:30.603035 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc85546-0e0b-4693-857d-2b687d759f31 service placement] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:24:30.608200 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36345ad-a568-4595-a44b-c8f6dc113efd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '857eb7657e7d45938baca17f91a16901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:24:30.608868 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36345ad-a568-4595-a44b-c8f6dc113efd service swift] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:24:30.609177 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36345ad-a568-4595-a44b-c8f6dc113efd service swift] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:24:30.609492 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36345ad-a568-4595-a44b-c8f6dc113efd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:24:30.618871 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36345ad-a568-4595-a44b-c8f6dc113efd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:24:30.620653 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36345ad-a568-4595-a44b-c8f6dc113efd service swift] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:24:30.630268 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1193/2412] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Sep 5 12:24:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 12:24:40.564533 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:24:40.626640 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=6 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:25:00.729516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12267749-9423-47bb-90d6-9ee943236f77 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:00.731860 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-12267749-9423-47bb-90d6-9ee943236f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:00.757157 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12267749-9423-47bb-90d6-9ee943236f77 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:00.766859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d3fc2-c0b4-466b-a736-dd21a0f475bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:00.767697 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d3fc2-c0b4-466b-a736-dd21a0f475bb service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:00.767979 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d3fc2-c0b4-466b-a736-dd21a0f475bb service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:00.768248 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d3fc2-c0b4-466b-a736-dd21a0f475bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:00.770660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-750d3fc2-c0b4-466b-a736-dd21a0f475bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:00.785451 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750d3fc2-c0b4-466b-a736-dd21a0f475bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:00.787130 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750d3fc2-c0b4-466b-a736-dd21a0f475bb service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:00.799375 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1220/2413] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:25:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:25:10.641943 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36345ad-a568-4595-a44b-c8f6dc113efd service swift] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:10.655945 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36345ad-a568-4595-a44b-c8f6dc113efd service swift] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:10.661668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c361e6-0c41-4741-ba77-fd157cea67ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:10.662268 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c361e6-0c41-4741-ba77-fd157cea67ad service nova] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:10.662575 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c361e6-0c41-4741-ba77-fd157cea67ad service nova] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:10.662850 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c361e6-0c41-4741-ba77-fd157cea67ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:10.664613 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-79c361e6-0c41-4741-ba77-fd157cea67ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:10.677511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c361e6-0c41-4741-ba77-fd157cea67ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:10.679128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c361e6-0c41-4741-ba77-fd157cea67ad service nova] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:10.692437 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1194/2414] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:25:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:25:10.793864 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=6 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:25:13.457301 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846e141-5a57-435b-b926-56d5d0988e70 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:13.457930 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846e141-5a57-435b-b926-56d5d0988e70 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:13.457930 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846e141-5a57-435b-b926-56d5d0988e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:13.479011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-b846e141-5a57-435b-b926-56d5d0988e70 None None] MFA Rules not processed for user `8c5ae98336c441118d01f46a50f732d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:25:13.480291 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b846e141-5a57-435b-b926-56d5d0988e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:13.491886 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1221/2415] 10.222.0.16 () {58 vars in 967 bytes} [Mon Sep 5 12:25:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:25:13.514737 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c361e6-0c41-4741-ba77-fd157cea67ad service nova] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:13.527449 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c361e6-0c41-4741-ba77-fd157cea67ad service nova] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:13.535688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04508d68-7d5a-4dbd-975d-3882b180d321 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '857eb7657e7d45938baca17f91a16901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:13.536443 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04508d68-7d5a-4dbd-975d-3882b180d321 service swift] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:13.536903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04508d68-7d5a-4dbd-975d-3882b180d321 service swift] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:13.537347 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04508d68-7d5a-4dbd-975d-3882b180d321 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:13.548738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04508d68-7d5a-4dbd-975d-3882b180d321 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:13.550713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04508d68-7d5a-4dbd-975d-3882b180d321 service swift] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:13.561800 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1195/2416] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Sep 5 12:25:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 12:25:23.472932 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:25:23.557665 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=12 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:25:31.198287 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846e141-5a57-435b-b926-56d5d0988e70 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:31.199798 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b846e141-5a57-435b-b926-56d5d0988e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:31.269213 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846e141-5a57-435b-b926-56d5d0988e70 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:31.273550 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e753c-c764-4f53-86c7-60309db5e71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:31.274172 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e753c-c764-4f53-86c7-60309db5e71e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:31.274452 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e753c-c764-4f53-86c7-60309db5e71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:31.274726 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e753c-c764-4f53-86c7-60309db5e71e admin admin] PATH_INFO: `/v3/users/4e187162f3e64acdb8cd24f660ceeca6` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:31.276045 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6e753c-c764-4f53-86c7-60309db5e71e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e187162f3e64acdb8cd24f660ceeca6)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:31.278744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6e753c-c764-4f53-86c7-60309db5e71e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:31.324258 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fb6e753c-c764-4f53-86c7-60309db5e71e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e187162f3e64acdb8cd24f660ceeca6'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:25:31.331910 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fb6e753c-c764-4f53-86c7-60309db5e71e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e187162f3e64acdb8cd24f660ceeca6'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:25:31.332598 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fb6e753c-c764-4f53-86c7-60309db5e71e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e187162f3e64acdb8cd24f660ceeca6'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:25:31.333244 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fb6e753c-c764-4f53-86c7-60309db5e71e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e187162f3e64acdb8cd24f660ceeca6'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:25:31.345979 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fb6e753c-c764-4f53-86c7-60309db5e71e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e187162f3e64acdb8cd24f660ceeca6'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:25:31.358798 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1222/2417] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:25:31 2022] DELETE /identity/v3/users/4e187162f3e64acdb8cd24f660ceeca6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:25:31.536197 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04508d68-7d5a-4dbd-975d-3882b180d321 service swift] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:31.546955 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04508d68-7d5a-4dbd-975d-3882b180d321 service swift] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:31.613591 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e6ca2-b6e5-4c46-b033-aaf15124db59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:31.614674 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e6ca2-b6e5-4c46-b033-aaf15124db59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:31.615881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e6ca2-b6e5-4c46-b033-aaf15124db59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:31.616630 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e6ca2-b6e5-4c46-b033-aaf15124db59 admin admin] PATH_INFO: `/v3/projects/57a93fc888a145c88b28b8b73060731e` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:31.619952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e6ca2-b6e5-4c46-b033-aaf15124db59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57a93fc888a145c88b28b8b73060731e)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:31.623178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e6ca2-b6e5-4c46-b033-aaf15124db59 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:31.670135 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b97e6ca2-b6e5-4c46-b033-aaf15124db59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57a93fc888a145c88b28b8b73060731e', 'request_id': 'req-b97e6ca2-b6e5-4c46-b033-aaf15124db59'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:25:31.677744 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b97e6ca2-b6e5-4c46-b033-aaf15124db59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57a93fc888a145c88b28b8b73060731e', 'request_id': 'req-b97e6ca2-b6e5-4c46-b033-aaf15124db59'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:25:31.683569 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b97e6ca2-b6e5-4c46-b033-aaf15124db59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57a93fc888a145c88b28b8b73060731e', 'request_id': 'req-b97e6ca2-b6e5-4c46-b033-aaf15124db59'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:25:31.695378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b97e6ca2-b6e5-4c46-b033-aaf15124db59 admin admin] The token cache is being invalidate because project 57a93fc888a145c88b28b8b73060731e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:25:31.695699 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b97e6ca2-b6e5-4c46-b033-aaf15124db59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:25:31.696365 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b97e6ca2-b6e5-4c46-b033-aaf15124db59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:25:31.697877 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1196/2418] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:25:31 2022] DELETE /identity/v3/projects/57a93fc888a145c88b28b8b73060731e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:25:31.703753 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f608486-38ef-4d3e-9bbd-7b5880b1d17d None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:31.704021 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f608486-38ef-4d3e-9bbd-7b5880b1d17d None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:31.704303 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f608486-38ef-4d3e-9bbd-7b5880b1d17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:31.724050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-5f608486-38ef-4d3e-9bbd-7b5880b1d17d None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:25:31.725269 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5f608486-38ef-4d3e-9bbd-7b5880b1d17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:31.788074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1223/2419] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:25:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:25:31.796024 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4f32d-f60b-4d9e-85a2-4219b3e31cf6 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:31.796303 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4f32d-f60b-4d9e-85a2-4219b3e31cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:31.796577 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4f32d-f60b-4d9e-85a2-4219b3e31cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:31.817000 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-3ea4f32d-f60b-4d9e-85a2-4219b3e31cf6 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:25:31.818221 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-3ea4f32d-f60b-4d9e-85a2-4219b3e31cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:31.826956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1197/2420] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:25:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:25:31.831700 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f608486-38ef-4d3e-9bbd-7b5880b1d17d None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:31.841615 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f608486-38ef-4d3e-9bbd-7b5880b1d17d None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:31.846241 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ad14c-6b40-4a21-9174-437c5e87c1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:31.846849 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ad14c-6b40-4a21-9174-437c5e87c1c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:31.847129 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ad14c-6b40-4a21-9174-437c5e87c1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:31.847401 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ad14c-6b40-4a21-9174-437c5e87c1c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:31.848093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ad14c-6b40-4a21-9174-437c5e87c1c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:25:31.848398 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ad14c-6b40-4a21-9174-437c5e87c1c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:31.850000 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ad14c-6b40-4a21-9174-437c5e87c1c8 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:31.862437 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1224/2421] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:25:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:25:31.866951 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4f32d-f60b-4d9e-85a2-4219b3e31cf6 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:31.877397 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4f32d-f60b-4d9e-85a2-4219b3e31cf6 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:31.882315 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431f22e-e720-40d4-81b8-a406edf9a70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:31.882931 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431f22e-e720-40d4-81b8-a406edf9a70a admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:31.883210 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431f22e-e720-40d4-81b8-a406edf9a70a admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:31.883483 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431f22e-e720-40d4-81b8-a406edf9a70a admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:31.884172 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e431f22e-e720-40d4-81b8-a406edf9a70a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:31.886092 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e431f22e-e720-40d4-81b8-a406edf9a70a admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:31.920469 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1198/2422] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:25:31 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:25:31.922291 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ad14c-6b40-4a21-9174-437c5e87c1c8 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:31.932383 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ad14c-6b40-4a21-9174-437c5e87c1c8 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:31.936649 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:25:31.994769 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec2177-98a2-408c-9c13-7f6631c1c1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:31.995381 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec2177-98a2-408c-9c13-7f6631c1c1ce admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:31.995705 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec2177-98a2-408c-9c13-7f6631c1c1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:31.995983 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec2177-98a2-408c-9c13-7f6631c1c1ce admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:31.997116 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dec2177-98a2-408c-9c13-7f6631c1c1ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:31.999020 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dec2177-98a2-408c-9c13-7f6631c1c1ce admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:32.022058 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1225/2423] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:25:31 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:25:32.026342 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431f22e-e720-40d4-81b8-a406edf9a70a admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:32.037793 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431f22e-e720-40d4-81b8-a406edf9a70a admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:32.044513 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf068e-8eb7-40bb-9941-504f97300f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:32.045072 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf068e-8eb7-40bb-9941-504f97300f44 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:32.045329 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf068e-8eb7-40bb-9941-504f97300f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:32.045616 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf068e-8eb7-40bb-9941-504f97300f44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:32.046355 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bf068e-8eb7-40bb-9941-504f97300f44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:32.047898 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bf068e-8eb7-40bb-9941-504f97300f44 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:32.060210 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1199/2424] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:25:32.064652 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec2177-98a2-408c-9c13-7f6631c1c1ce admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:32.075739 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec2177-98a2-408c-9c13-7f6631c1c1ce admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:32.080458 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f5a1a-97d6-4cba-819d-ddb265cb7fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:32.081065 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f5a1a-97d6-4cba-819d-ddb265cb7fac admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:32.081342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f5a1a-97d6-4cba-819d-ddb265cb7fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:32.081615 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f5a1a-97d6-4cba-819d-ddb265cb7fac admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:32.082261 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4f5a1a-97d6-4cba-819d-ddb265cb7fac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:32.083892 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4f5a1a-97d6-4cba-819d-ddb265cb7fac admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:32.095576 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1226/2425] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:25:32.100136 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf068e-8eb7-40bb-9941-504f97300f44 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:32.112331 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf068e-8eb7-40bb-9941-504f97300f44 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:32.118048 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1007f6-133c-413d-9c3c-cf652575c744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:32.118703 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1007f6-133c-413d-9c3c-cf652575c744 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:32.118982 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1007f6-133c-413d-9c3c-cf652575c744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:32.119259 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1007f6-133c-413d-9c3c-cf652575c744 admin admin] PATH_INFO: `/v3/projects/4d5cb05cfc60481fa4144ee7b077ca4f/users/ebd8a00f21344c7990ffe1a7ff2986ef/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:32.136471 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1007f6-133c-413d-9c3c-cf652575c744 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d5cb05cfc60481fa4144ee7b077ca4f, user_id=ebd8a00f21344c7990ffe1a7ff2986ef, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:32.139210 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1007f6-133c-413d-9c3c-cf652575c744 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:32.154059 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1200/2426] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:25:32 2022] PUT /identity/v3/projects/4d5cb05cfc60481fa4144ee7b077ca4f/users/ebd8a00f21344c7990ffe1a7ff2986ef/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:25:32.159279 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ed6fa-bab0-4a08-af24-76b443af5d25 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:32.159602 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ed6fa-bab0-4a08-af24-76b443af5d25 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:32.159877 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ed6fa-bab0-4a08-af24-76b443af5d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:32.185781 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-a64ed6fa-bab0-4a08-af24-76b443af5d25 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:25:32.187337 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a64ed6fa-bab0-4a08-af24-76b443af5d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:32.250931 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1227/2427] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:25:32.260500 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1007f6-133c-413d-9c3c-cf652575c744 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:32.262219 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-dc1007f6-133c-413d-9c3c-cf652575c744 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:32.297360 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:25:32.326729 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1007f6-133c-413d-9c3c-cf652575c744 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:32.330341 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b605c01-00eb-44e4-a5e5-318be7b5e5c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:32.330947 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b605c01-00eb-44e4-a5e5-318be7b5e5c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:32.331227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b605c01-00eb-44e4-a5e5-318be7b5e5c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:32.331511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b605c01-00eb-44e4-a5e5-318be7b5e5c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:32.340409 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b605c01-00eb-44e4-a5e5-318be7b5e5c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:32.341842 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b605c01-00eb-44e4-a5e5-318be7b5e5c6 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:32.351127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1201/2428] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:25:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:25:36.445422 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d042522-fa18-431e-bcae-dbc5625ab433 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:36.445961 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d042522-fa18-431e-bcae-dbc5625ab433 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:36.445961 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d042522-fa18-431e-bcae-dbc5625ab433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:36.482305 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-4d042522-fa18-431e-bcae-dbc5625ab433 None None] MFA Rules not processed for user `ebd8a00f21344c7990ffe1a7ff2986ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:25:36.483629 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4d042522-fa18-431e-bcae-dbc5625ab433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:36.585046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1228/2429] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:25:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:25:36.589821 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b605c01-00eb-44e4-a5e5-318be7b5e5c6 service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:36.600108 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b605c01-00eb-44e4-a5e5-318be7b5e5c6 service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:36.605261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a50286-225c-469b-a579-9649c1df0802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:36.606416 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a50286-225c-469b-a579-9649c1df0802 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:36.606694 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a50286-225c-469b-a579-9649c1df0802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:36.606964 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a50286-225c-469b-a579-9649c1df0802 admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:36.607687 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a50286-225c-469b-a579-9649c1df0802 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:36.609641 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a50286-225c-469b-a579-9649c1df0802 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:36.643264 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1202/2430] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:25:36 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:25:36.644643 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d042522-fa18-431e-bcae-dbc5625ab433 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:36.656376 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d042522-fa18-431e-bcae-dbc5625ab433 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:36.669003 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:25:36.717160 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ac8c28-e2cd-4266-8973-8d88d3e54078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:36.717777 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac8c28-e2cd-4266-8973-8d88d3e54078 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:36.718058 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac8c28-e2cd-4266-8973-8d88d3e54078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:36.718338 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac8c28-e2cd-4266-8973-8d88d3e54078 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:36.718978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ac8c28-e2cd-4266-8973-8d88d3e54078 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:36.720943 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ac8c28-e2cd-4266-8973-8d88d3e54078 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:36.746019 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1229/2431] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:25:36 2022] POST /identity/v3/users => generated 295 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:25:36.750842 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a50286-225c-469b-a579-9649c1df0802 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:36.761541 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a50286-225c-469b-a579-9649c1df0802 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:36.766668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880db45-459c-47f0-a597-0c5d4598b321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:36.767296 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880db45-459c-47f0-a597-0c5d4598b321 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:36.767595 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880db45-459c-47f0-a597-0c5d4598b321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:36.767865 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880db45-459c-47f0-a597-0c5d4598b321 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:36.768733 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9880db45-459c-47f0-a597-0c5d4598b321 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:36.770374 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9880db45-459c-47f0-a597-0c5d4598b321 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:36.784988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1203/2432] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:25:36.789262 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ac8c28-e2cd-4266-8973-8d88d3e54078 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:36.800693 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ac8c28-e2cd-4266-8973-8d88d3e54078 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:36.805519 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da59aeb-d5b0-4813-a534-49a63d58d916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:36.806193 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da59aeb-d5b0-4813-a534-49a63d58d916 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:36.806481 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da59aeb-d5b0-4813-a534-49a63d58d916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:36.806744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da59aeb-d5b0-4813-a534-49a63d58d916 admin admin] PATH_INFO: `/v3/projects/0f6efe7cbc5e4b499f13dc0ccf6a67d3/users/3713e2b332654d49998d42ac8cd482e6/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:36.823776 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da59aeb-d5b0-4813-a534-49a63d58d916 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f6efe7cbc5e4b499f13dc0ccf6a67d3, user_id=3713e2b332654d49998d42ac8cd482e6, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:36.826468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da59aeb-d5b0-4813-a534-49a63d58d916 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:36.840671 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1230/2433] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:25:36 2022] PUT /identity/v3/projects/0f6efe7cbc5e4b499f13dc0ccf6a67d3/users/3713e2b332654d49998d42ac8cd482e6/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:25:36.844840 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880db45-459c-47f0-a597-0c5d4598b321 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:36.857343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880db45-459c-47f0-a597-0c5d4598b321 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:36.918125 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9654206-e8a4-4d8b-a8ef-3cfb213af43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:36.918756 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9654206-e8a4-4d8b-a8ef-3cfb213af43d admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:36.919039 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9654206-e8a4-4d8b-a8ef-3cfb213af43d admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:36.919309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9654206-e8a4-4d8b-a8ef-3cfb213af43d admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:36.920722 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9654206-e8a4-4d8b-a8ef-3cfb213af43d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:36.922410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9654206-e8a4-4d8b-a8ef-3cfb213af43d admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:36.933826 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1204/2434] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:25:36.937907 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da59aeb-d5b0-4813-a534-49a63d58d916 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:36.958620 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da59aeb-d5b0-4813-a534-49a63d58d916 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:36.966007 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc272d0c-2706-4093-b12b-dc2ba451631d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:36.966965 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc272d0c-2706-4093-b12b-dc2ba451631d admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:36.967344 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc272d0c-2706-4093-b12b-dc2ba451631d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:36.967752 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc272d0c-2706-4093-b12b-dc2ba451631d admin admin] PATH_INFO: `/v3/projects/0f6efe7cbc5e4b499f13dc0ccf6a67d3/users/3713e2b332654d49998d42ac8cd482e6/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:36.969993 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc272d0c-2706-4093-b12b-dc2ba451631d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f6efe7cbc5e4b499f13dc0ccf6a67d3, user_id=3713e2b332654d49998d42ac8cd482e6, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:36.976753 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc272d0c-2706-4093-b12b-dc2ba451631d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:36.993923 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1231/2435] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:25:36 2022] PUT /identity/v3/projects/0f6efe7cbc5e4b499f13dc0ccf6a67d3/users/3713e2b332654d49998d42ac8cd482e6/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:25:41.294918 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095425c-1f4a-48f8-b266-c9121888258b None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:41.295547 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095425c-1f4a-48f8-b266-c9121888258b None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:41.295547 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095425c-1f4a-48f8-b266-c9121888258b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:41.329222 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-9095425c-1f4a-48f8-b266-c9121888258b None None] MFA Rules not processed for user `3713e2b332654d49998d42ac8cd482e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:25:41.331003 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-9095425c-1f4a-48f8-b266-c9121888258b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:41.405237 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1205/2436] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:25:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:25:41.423291 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc272d0c-2706-4093-b12b-dc2ba451631d admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:41.425299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-dc272d0c-2706-4093-b12b-dc2ba451631d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:41.519872 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc272d0c-2706-4093-b12b-dc2ba451631d admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:41.526306 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd20ba6-8da3-49e3-9020-fbe7b08e606b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:41.528407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd20ba6-8da3-49e3-9020-fbe7b08e606b service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:41.528407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd20ba6-8da3-49e3-9020-fbe7b08e606b service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:41.528407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd20ba6-8da3-49e3-9020-fbe7b08e606b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:41.591272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd20ba6-8da3-49e3-9020-fbe7b08e606b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:41.592970 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd20ba6-8da3-49e3-9020-fbe7b08e606b service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:41.602900 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1232/2437] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:25:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:25:48.152327 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9095425c-1f4a-48f8-b266-c9121888258b None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:48.165588 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9095425c-1f4a-48f8-b266-c9121888258b None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:48.171610 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed85afc-0c4e-4d7e-8c5a-b0ca6383b002 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:48.172199 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed85afc-0c4e-4d7e-8c5a-b0ca6383b002 service nova] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:48.172475 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed85afc-0c4e-4d7e-8c5a-b0ca6383b002 service nova] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:48.172742 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed85afc-0c4e-4d7e-8c5a-b0ca6383b002 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:48.174441 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-9ed85afc-0c4e-4d7e-8c5a-b0ca6383b002 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:48.190030 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:25:48.238443 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed85afc-0c4e-4d7e-8c5a-b0ca6383b002 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:48.240004 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed85afc-0c4e-4d7e-8c5a-b0ca6383b002 service nova] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:48.249527 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1206/2438] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:25:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:25:51.598816 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=93,keystone:INSERT=5 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:25:57.841598 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd20ba6-8da3-49e3-9020-fbe7b08e606b service nova] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:57.843746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7dd20ba6-8da3-49e3-9020-fbe7b08e606b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:57.900810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd20ba6-8da3-49e3-9020-fbe7b08e606b service nova] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:57.905814 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fbdf8f-e449-49f5-83c5-378099b1adc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:57.906423 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbdf8f-e449-49f5-83c5-378099b1adc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:57.906725 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbdf8f-e449-49f5-83c5-378099b1adc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:57.906997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbdf8f-e449-49f5-83c5-378099b1adc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:57.909003 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-48fbdf8f-e449-49f5-83c5-378099b1adc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:57.972420 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fbdf8f-e449-49f5-83c5-378099b1adc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:57.973870 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fbdf8f-e449-49f5-83c5-378099b1adc1 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:57.984306 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1233/2439] 10.222.0.16 () {60 vars in 1396 bytes} [Mon Sep 5 12:25:57 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3268 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:25:58.245116 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=20 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:26:07.618046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35e73f-4794-433c-a8c7-c4bb8493767e None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:26:07.618706 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35e73f-4794-433c-a8c7-c4bb8493767e None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:26:07.618706 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35e73f-4794-433c-a8c7-c4bb8493767e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:26:07.648129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-3f35e73f-4794-433c-a8c7-c4bb8493767e None None] MFA Rules not processed for user `ebd8a00f21344c7990ffe1a7ff2986ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:26:07.649375 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-3f35e73f-4794-433c-a8c7-c4bb8493767e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:26:07.656785 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1207/2440] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:26:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:26:07.979778 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=48 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:26:11.597185 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52dad92-1bfc-4c7c-88ee-aec20e0c691c None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:26:11.597841 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52dad92-1bfc-4c7c-88ee-aec20e0c691c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:26:11.597940 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52dad92-1bfc-4c7c-88ee-aec20e0c691c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:26:11.618854 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-e52dad92-1bfc-4c7c-88ee-aec20e0c691c None None] MFA Rules not processed for user `ff0999bd21bf4db5981053922200be2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:26:11.620332 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e52dad92-1bfc-4c7c-88ee-aec20e0c691c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:26:11.628424 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1234/2441] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:26:11 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:26:17.640330 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:26:21.612995 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:26:32.938913 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35e73f-4794-433c-a8c7-c4bb8493767e None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:26:32.949053 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35e73f-4794-433c-a8c7-c4bb8493767e None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:26:32.952672 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab58db-9c46-4695-8359-fe8e7a564ca6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:26:32.953276 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab58db-9c46-4695-8359-fe8e7a564ca6 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:26:32.953555 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab58db-9c46-4695-8359-fe8e7a564ca6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:26:32.953826 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab58db-9c46-4695-8359-fe8e7a564ca6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:26:32.964541 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab58db-9c46-4695-8359-fe8e7a564ca6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:26:32.965998 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab58db-9c46-4695-8359-fe8e7a564ca6 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:26:32.976289 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1208/2442] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:26:32 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:26:41.748237 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde4273-ad9e-4d17-a3f6-49597529b4ff None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:26:41.748904 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde4273-ad9e-4d17-a3f6-49597529b4ff None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:26:41.748904 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde4273-ad9e-4d17-a3f6-49597529b4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:26:41.779893 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-4cde4273-ad9e-4d17-a3f6-49597529b4ff None None] MFA Rules not processed for user `83d3b245c185424a9b6e5d69d691f031`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:26:41.781145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4cde4273-ad9e-4d17-a3f6-49597529b4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:26:41.857248 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1235/2443] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:26:41 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:26:42.971917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=6 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:26:51.854846 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=37 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:26:55.257610 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab58db-9c46-4695-8359-fe8e7a564ca6 service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:26:55.259015 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-f7ab58db-9c46-4695-8359-fe8e7a564ca6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:26:55.393368 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab58db-9c46-4695-8359-fe8e7a564ca6 service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:26:55.399362 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb89cc-3fdc-448a-be92-0a84ca8fafb5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524bcfffcdfa4790885e0d4cbd66fa80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:26:55.400448 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb89cc-3fdc-448a-be92-0a84ca8fafb5 service placement] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:26:55.400448 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb89cc-3fdc-448a-be92-0a84ca8fafb5 service placement] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:26:55.400611 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb89cc-3fdc-448a-be92-0a84ca8fafb5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:26:55.403322 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-d3cb89cc-3fdc-448a-be92-0a84ca8fafb5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:26:55.416724 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cb89cc-3fdc-448a-be92-0a84ca8fafb5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:26:55.418354 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cb89cc-3fdc-448a-be92-0a84ca8fafb5 service placement] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:26:55.428966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1209/2444] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:26:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:26:57.828609 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde4273-ad9e-4d17-a3f6-49597529b4ff None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:26:57.865977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde4273-ad9e-4d17-a3f6-49597529b4ff None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:26:57.871612 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2cad4-571f-4fd6-a1d7-6de4d2640fe8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:26:57.872243 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2cad4-571f-4fd6-a1d7-6de4d2640fe8 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:26:57.872618 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2cad4-571f-4fd6-a1d7-6de4d2640fe8 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:26:57.872895 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2cad4-571f-4fd6-a1d7-6de4d2640fe8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:26:57.874695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ece2cad4-571f-4fd6-a1d7-6de4d2640fe8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:26:57.908537 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2cad4-571f-4fd6-a1d7-6de4d2640fe8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:26:57.910255 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2cad4-571f-4fd6-a1d7-6de4d2640fe8 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:26:57.925629 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1236/2445] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:26:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:27:05.424334 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=59 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:27:07.920701 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=18 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:27:19.080477 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb89cc-3fdc-448a-be92-0a84ca8fafb5 service placement] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:19.092726 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb89cc-3fdc-448a-be92-0a84ca8fafb5 service placement] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:19.098311 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da295c5-ac80-4dfc-8583-9e03a3b9f2f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:19.098922 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da295c5-ac80-4dfc-8583-9e03a3b9f2f4 service nova] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:19.099227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da295c5-ac80-4dfc-8583-9e03a3b9f2f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:19.099500 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da295c5-ac80-4dfc-8583-9e03a3b9f2f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:19.101771 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-6da295c5-ac80-4dfc-8583-9e03a3b9f2f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:27:19.161332 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da295c5-ac80-4dfc-8583-9e03a3b9f2f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:19.163618 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da295c5-ac80-4dfc-8583-9e03a3b9f2f4 service nova] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:19.173419 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1210/2446] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:27:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:27:26.553536 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2cad4-571f-4fd6-a1d7-6de4d2640fe8 service nova] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:26.573162 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2cad4-571f-4fd6-a1d7-6de4d2640fe8 service nova] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:26.577073 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c314ff2-107b-4b2e-a6c0-5c49552a589e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:26.577676 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c314ff2-107b-4b2e-a6c0-5c49552a589e service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:26.577954 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c314ff2-107b-4b2e-a6c0-5c49552a589e service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:26.578227 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c314ff2-107b-4b2e-a6c0-5c49552a589e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:26.579987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8c314ff2-107b-4b2e-a6c0-5c49552a589e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:27:26.603473 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c314ff2-107b-4b2e-a6c0-5c49552a589e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:26.604929 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c314ff2-107b-4b2e-a6c0-5c49552a589e service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:26.612819 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1237/2447] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:27:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:27:26.738582 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3087e2-0ad0-48af-a48c-c99fd1bb7aa9 None None] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:26.739220 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3087e2-0ad0-48af-a48c-c99fd1bb7aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:26.739220 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3087e2-0ad0-48af-a48c-c99fd1bb7aa9 None None] PATH_INFO: `/` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:26.740563 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1211/2448] 10.222.0.16 () {54 vars in 921 bytes} [Mon Sep 5 12:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:27:26.747499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f5e04-1b21-471d-a79d-c449a92290f7 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:26.747818 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f5e04-1b21-471d-a79d-c449a92290f7 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:26.748121 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f5e04-1b21-471d-a79d-c449a92290f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:26.792584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-257f5e04-1b21-471d-a79d-c449a92290f7 None None] MFA Rules not processed for user `9f79599ed0b74bdbab198627ef72252a`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:27:26.793802 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-257f5e04-1b21-471d-a79d-c449a92290f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:27:26.802093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1238/2449] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Sep 5 12:27:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:27:26.815351 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3087e2-0ad0-48af-a48c-c99fd1bb7aa9 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:26.827208 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3087e2-0ad0-48af-a48c-c99fd1bb7aa9 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:26.831067 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04bff7-0f57-4715-8efc-3f54d495e30f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:26.831738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04bff7-0f57-4715-8efc-3f54d495e30f service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:26.832152 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04bff7-0f57-4715-8efc-3f54d495e30f service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:26.832435 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04bff7-0f57-4715-8efc-3f54d495e30f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:26.844195 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04bff7-0f57-4715-8efc-3f54d495e30f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:26.846032 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04bff7-0f57-4715-8efc-3f54d495e30f service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:26.856373 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1212/2450] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:27:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:27:26.940592 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307fafd-9e45-4064-8f0a-6bae03ac326a None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:26.940872 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307fafd-9e45-4064-8f0a-6bae03ac326a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:26.941146 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307fafd-9e45-4064-8f0a-6bae03ac326a None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:26.942487 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1239/2451] 10.222.0.16 () {54 vars in 921 bytes} [Mon Sep 5 12:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:27:26.947464 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd0fa2-4a84-430f-a8ef-3fabf07530eb None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:26.947965 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd0fa2-4a84-430f-a8ef-3fabf07530eb None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:26.948089 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd0fa2-4a84-430f-a8ef-3fabf07530eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:26.971664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-43bd0fa2-4a84-430f-a8ef-3fabf07530eb None None] MFA Rules not processed for user `7cda290418504f0db85d665479a05353`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:27:26.973416 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-43bd0fa2-4a84-430f-a8ef-3fabf07530eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:27:26.988126 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1213/2452] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Sep 5 12:27:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:27:27.002126 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8307fafd-9e45-4064-8f0a-6bae03ac326a None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:27.014568 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8307fafd-9e45-4064-8f0a-6bae03ac326a None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:27.018627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371fd7e4-3a12-4b7b-9b52-fa5c92515e35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:27.019216 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fd7e4-3a12-4b7b-9b52-fa5c92515e35 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:27.019494 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fd7e4-3a12-4b7b-9b52-fa5c92515e35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:27.019791 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fd7e4-3a12-4b7b-9b52-fa5c92515e35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:27.031146 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371fd7e4-3a12-4b7b-9b52-fa5c92515e35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:27.032589 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371fd7e4-3a12-4b7b-9b52-fa5c92515e35 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:27.045634 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1240/2453] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:27:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:27:36.965004 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=37 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:27:37.039785 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=32 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:27:37.227263 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a0479-84d5-4f78-917f-cc80b1b50d7c None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:37.227549 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a0479-84d5-4f78-917f-cc80b1b50d7c None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:37.227830 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a0479-84d5-4f78-917f-cc80b1b50d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:37.248453 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-208a0479-84d5-4f78-917f-cc80b1b50d7c None None] MFA Rules not processed for user `45bd61574c6c4ebb9d6fe0c87228065f`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:27:37.249703 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-208a0479-84d5-4f78-917f-cc80b1b50d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:27:37.257134 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1214/2454] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:27:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:27:47.241501 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:27:49.863459 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371fd7e4-3a12-4b7b-9b52-fa5c92515e35 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:49.872416 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371fd7e4-3a12-4b7b-9b52-fa5c92515e35 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:49.876401 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086966c3-e9a7-436a-9233-21beea8ace85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:49.877001 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086966c3-e9a7-436a-9233-21beea8ace85 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:49.877283 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086966c3-e9a7-436a-9233-21beea8ace85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:49.877557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086966c3-e9a7-436a-9233-21beea8ace85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:49.879310 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-086966c3-e9a7-436a-9233-21beea8ace85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:27:49.967367 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086966c3-e9a7-436a-9233-21beea8ace85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:49.968836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086966c3-e9a7-436a-9233-21beea8ace85 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:49.987577 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1241/2455] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:27:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:27:53.289918 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208a0479-84d5-4f78-917f-cc80b1b50d7c None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:53.302308 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208a0479-84d5-4f78-917f-cc80b1b50d7c None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:53.307715 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f7b19-8419-4795-9d0d-2c680f26bcf6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524bcfffcdfa4790885e0d4cbd66fa80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:53.308459 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f7b19-8419-4795-9d0d-2c680f26bcf6 service placement] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:53.308837 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f7b19-8419-4795-9d0d-2c680f26bcf6 service placement] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:53.309226 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f7b19-8419-4795-9d0d-2c680f26bcf6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:53.311085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-c69f7b19-8419-4795-9d0d-2c680f26bcf6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:27:53.323433 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69f7b19-8419-4795-9d0d-2c680f26bcf6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:53.326298 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69f7b19-8419-4795-9d0d-2c680f26bcf6 service placement] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:53.341107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1215/2456] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:27:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:27:53.386740 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086966c3-e9a7-436a-9233-21beea8ace85 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:53.388356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-086966c3-e9a7-436a-9233-21beea8ace85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:27:53.404544 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086966c3-e9a7-436a-9233-21beea8ace85 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:53.410258 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bfeb61-e0af-43b3-840a-38ff89309fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:53.411184 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfeb61-e0af-43b3-840a-38ff89309fe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:53.411707 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfeb61-e0af-43b3-840a-38ff89309fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:53.412206 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfeb61-e0af-43b3-840a-38ff89309fe0 admin admin] PATH_INFO: `/v3/users/ff0999bd21bf4db5981053922200be2a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:53.414385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bfeb61-e0af-43b3-840a-38ff89309fe0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff0999bd21bf4db5981053922200be2a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:53.416991 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bfeb61-e0af-43b3-840a-38ff89309fe0 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:53.474240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-15bfeb61-e0af-43b3-840a-38ff89309fe0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff0999bd21bf4db5981053922200be2a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:27:53.484900 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-15bfeb61-e0af-43b3-840a-38ff89309fe0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff0999bd21bf4db5981053922200be2a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:27:53.485687 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-15bfeb61-e0af-43b3-840a-38ff89309fe0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff0999bd21bf4db5981053922200be2a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:27:53.486393 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-15bfeb61-e0af-43b3-840a-38ff89309fe0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff0999bd21bf4db5981053922200be2a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:27:53.501278 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-15bfeb61-e0af-43b3-840a-38ff89309fe0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff0999bd21bf4db5981053922200be2a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:27:53.515224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1242/2457] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:27:53 2022] DELETE /identity/v3/users/ff0999bd21bf4db5981053922200be2a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:27:53.688265 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f7b19-8419-4795-9d0d-2c680f26bcf6 service placement] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:53.699722 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f7b19-8419-4795-9d0d-2c680f26bcf6 service placement] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:53.763561 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fbf411-85a4-427c-97aa-02dc68c8ddc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:53.764245 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbf411-85a4-427c-97aa-02dc68c8ddc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:53.764640 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbf411-85a4-427c-97aa-02dc68c8ddc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:53.765017 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbf411-85a4-427c-97aa-02dc68c8ddc2 admin admin] PATH_INFO: `/v3/projects/938f765828c545a8840a6983bc5d9e6d` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:53.766379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fbf411-85a4-427c-97aa-02dc68c8ddc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=938f765828c545a8840a6983bc5d9e6d)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:53.768779 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fbf411-85a4-427c-97aa-02dc68c8ddc2 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:53.824402 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-d0fbf411-85a4-427c-97aa-02dc68c8ddc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '938f765828c545a8840a6983bc5d9e6d', 'request_id': 'req-d0fbf411-85a4-427c-97aa-02dc68c8ddc2'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:27:53.833028 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-d0fbf411-85a4-427c-97aa-02dc68c8ddc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '938f765828c545a8840a6983bc5d9e6d', 'request_id': 'req-d0fbf411-85a4-427c-97aa-02dc68c8ddc2'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:27:53.840328 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-d0fbf411-85a4-427c-97aa-02dc68c8ddc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '938f765828c545a8840a6983bc5d9e6d', 'request_id': 'req-d0fbf411-85a4-427c-97aa-02dc68c8ddc2'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:27:53.852259 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-d0fbf411-85a4-427c-97aa-02dc68c8ddc2 admin admin] The token cache is being invalidate because project 938f765828c545a8840a6983bc5d9e6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:27:53.852550 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-d0fbf411-85a4-427c-97aa-02dc68c8ddc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:27:53.853211 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-d0fbf411-85a4-427c-97aa-02dc68c8ddc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:27:53.854706 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1216/2458] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:27:53 2022] DELETE /identity/v3/projects/938f765828c545a8840a6983bc5d9e6d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:27:53.859739 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd96df-7b8c-4aa5-a30c-c35d000dd2b6 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:53.860021 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd96df-7b8c-4aa5-a30c-c35d000dd2b6 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:53.860298 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd96df-7b8c-4aa5-a30c-c35d000dd2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:53.918087 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-cdbd96df-7b8c-4aa5-a30c-c35d000dd2b6 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:27:53.919337 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-cdbd96df-7b8c-4aa5-a30c-c35d000dd2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:27:53.931572 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:27:53.984590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1243/2459] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:27:53.992992 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369c222-5b7e-45a3-9f6a-a18a536796ce None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:53.993284 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369c222-5b7e-45a3-9f6a-a18a536796ce None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:53.993553 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369c222-5b7e-45a3-9f6a-a18a536796ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:54.013833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-a369c222-5b7e-45a3-9f6a-a18a536796ce None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:27:54.014965 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-a369c222-5b7e-45a3-9f6a-a18a536796ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:27:54.022835 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1217/2460] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:27:54.027065 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd96df-7b8c-4aa5-a30c-c35d000dd2b6 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:54.036930 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd96df-7b8c-4aa5-a30c-c35d000dd2b6 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:54.041493 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8733a3-1c16-495d-a752-afa73b02e211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:54.042080 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8733a3-1c16-495d-a752-afa73b02e211 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:54.042356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8733a3-1c16-495d-a752-afa73b02e211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:54.042628 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8733a3-1c16-495d-a752-afa73b02e211 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:54.043263 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8733a3-1c16-495d-a752-afa73b02e211 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:27:54.043589 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8733a3-1c16-495d-a752-afa73b02e211 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:54.045169 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8733a3-1c16-495d-a752-afa73b02e211 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:54.057001 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1244/2461] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:27:54.061334 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369c222-5b7e-45a3-9f6a-a18a536796ce None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:54.071224 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369c222-5b7e-45a3-9f6a-a18a536796ce None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:54.075850 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23305c29-1683-4113-bc96-80da60c64f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:54.076439 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23305c29-1683-4113-bc96-80da60c64f67 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:54.076714 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23305c29-1683-4113-bc96-80da60c64f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:54.076983 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23305c29-1683-4113-bc96-80da60c64f67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:54.077606 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23305c29-1683-4113-bc96-80da60c64f67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:54.079487 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23305c29-1683-4113-bc96-80da60c64f67 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:54.108209 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1218/2462] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:27:54 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:27:54.112466 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8733a3-1c16-495d-a752-afa73b02e211 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:54.122832 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8733a3-1c16-495d-a752-afa73b02e211 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:54.178335 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f38984-6a61-4b08-8085-869577746369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:54.178936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f38984-6a61-4b08-8085-869577746369 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:54.179741 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f38984-6a61-4b08-8085-869577746369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:54.180011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f38984-6a61-4b08-8085-869577746369 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:54.180677 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f38984-6a61-4b08-8085-869577746369 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:54.182552 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f38984-6a61-4b08-8085-869577746369 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:54.203836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1245/2463] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:27:54 2022] POST /identity/v3/users => generated 282 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:27:54.208060 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23305c29-1683-4113-bc96-80da60c64f67 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:54.218447 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23305c29-1683-4113-bc96-80da60c64f67 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:54.223100 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227c603-db2f-4100-91ea-03a70fe1d176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:54.223741 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227c603-db2f-4100-91ea-03a70fe1d176 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:54.224024 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227c603-db2f-4100-91ea-03a70fe1d176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:54.224300 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227c603-db2f-4100-91ea-03a70fe1d176 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:54.224946 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c227c603-db2f-4100-91ea-03a70fe1d176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:54.227047 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c227c603-db2f-4100-91ea-03a70fe1d176 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:54.237572 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1219/2464] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:27:54.242413 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f38984-6a61-4b08-8085-869577746369 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:54.253087 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f38984-6a61-4b08-8085-869577746369 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:54.257693 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8eec8-cbcd-4da8-9bce-08155f775cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:54.258273 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8eec8-cbcd-4da8-9bce-08155f775cad admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:54.258548 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8eec8-cbcd-4da8-9bce-08155f775cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:54.258820 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8eec8-cbcd-4da8-9bce-08155f775cad admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:54.259461 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd8eec8-cbcd-4da8-9bce-08155f775cad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:54.261063 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd8eec8-cbcd-4da8-9bce-08155f775cad admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:54.271792 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1246/2465] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:27:54.275875 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227c603-db2f-4100-91ea-03a70fe1d176 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:54.286516 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227c603-db2f-4100-91ea-03a70fe1d176 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:54.291446 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096d757-a90d-4396-b615-e0460d72a919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:54.292135 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096d757-a90d-4396-b615-e0460d72a919 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:54.292417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096d757-a90d-4396-b615-e0460d72a919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:54.292692 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096d757-a90d-4396-b615-e0460d72a919 admin admin] PATH_INFO: `/v3/projects/35d8d6a3514b4760b7a4d891e3395f15/users/899cdca5b777419ab86c68ff76aeac83/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:54.307964 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c096d757-a90d-4396-b615-e0460d72a919 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35d8d6a3514b4760b7a4d891e3395f15, user_id=899cdca5b777419ab86c68ff76aeac83, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:54.310618 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c096d757-a90d-4396-b615-e0460d72a919 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:54.323678 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1220/2466] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:27:54 2022] PUT /identity/v3/projects/35d8d6a3514b4760b7a4d891e3395f15/users/899cdca5b777419ab86c68ff76aeac83/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:27:54.333852 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84041a86-9af4-4fe6-9337-b3ae8f3423a4 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:54.334143 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84041a86-9af4-4fe6-9337-b3ae8f3423a4 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:54.334419 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84041a86-9af4-4fe6-9337-b3ae8f3423a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:54.363078 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-84041a86-9af4-4fe6-9337-b3ae8f3423a4 None None] MFA Rules not processed for user `899cdca5b777419ab86c68ff76aeac83`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:27:54.364378 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-84041a86-9af4-4fe6-9337-b3ae8f3423a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:27:54.436075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1247/2467] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:27:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:27:54.440639 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096d757-a90d-4396-b615-e0460d72a919 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:54.453931 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:27:54.504929 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096d757-a90d-4396-b615-e0460d72a919 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:54.509721 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e924259-9bfd-4f4e-8004-3d3cdfc9c148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:54.510326 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e924259-9bfd-4f4e-8004-3d3cdfc9c148 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:54.510609 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e924259-9bfd-4f4e-8004-3d3cdfc9c148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:54.510882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e924259-9bfd-4f4e-8004-3d3cdfc9c148 admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:54.511542 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e924259-9bfd-4f4e-8004-3d3cdfc9c148 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:54.515229 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e924259-9bfd-4f4e-8004-3d3cdfc9c148 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:54.542151 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1221/2468] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:27:54 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:27:54.546421 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84041a86-9af4-4fe6-9337-b3ae8f3423a4 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:54.557811 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84041a86-9af4-4fe6-9337-b3ae8f3423a4 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:54.566729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:27:54.621109 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7efa1-439a-41de-8e1c-12e0b5518a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:54.621826 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7efa1-439a-41de-8e1c-12e0b5518a61 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:54.622203 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7efa1-439a-41de-8e1c-12e0b5518a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:54.622484 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7efa1-439a-41de-8e1c-12e0b5518a61 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:54.623208 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d7efa1-439a-41de-8e1c-12e0b5518a61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:54.625258 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d7efa1-439a-41de-8e1c-12e0b5518a61 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:54.648627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1248/2469] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:27:54 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:27:54.652835 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e924259-9bfd-4f4e-8004-3d3cdfc9c148 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:54.664249 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e924259-9bfd-4f4e-8004-3d3cdfc9c148 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:54.668995 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3272f39-6b1a-4c2d-8474-679c8b2ec457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:54.669599 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3272f39-6b1a-4c2d-8474-679c8b2ec457 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:54.669877 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3272f39-6b1a-4c2d-8474-679c8b2ec457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:54.670149 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3272f39-6b1a-4c2d-8474-679c8b2ec457 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:54.670809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3272f39-6b1a-4c2d-8474-679c8b2ec457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:54.672413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3272f39-6b1a-4c2d-8474-679c8b2ec457 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:54.684999 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1222/2470] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:27:54.689297 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7efa1-439a-41de-8e1c-12e0b5518a61 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:54.699346 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7efa1-439a-41de-8e1c-12e0b5518a61 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:54.704789 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e2a23-8107-4acf-9393-3c585c3cc5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:54.705445 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e2a23-8107-4acf-9393-3c585c3cc5bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:54.705734 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e2a23-8107-4acf-9393-3c585c3cc5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:54.706011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e2a23-8107-4acf-9393-3c585c3cc5bc admin admin] PATH_INFO: `/v3/projects/27e74b6a9b5e4394ae45a2e708319326/users/2a5b0c3a5e4843bba3a5da8ce25b4529/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:54.722258 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e2a23-8107-4acf-9393-3c585c3cc5bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=27e74b6a9b5e4394ae45a2e708319326, user_id=2a5b0c3a5e4843bba3a5da8ce25b4529, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:54.725004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e2a23-8107-4acf-9393-3c585c3cc5bc admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:54.741705 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1249/2471] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:27:54 2022] PUT /identity/v3/projects/27e74b6a9b5e4394ae45a2e708319326/users/2a5b0c3a5e4843bba3a5da8ce25b4529/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:27:54.745837 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3272f39-6b1a-4c2d-8474-679c8b2ec457 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:54.756592 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3272f39-6b1a-4c2d-8474-679c8b2ec457 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:54.818337 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad415063-36b0-4d72-ba4a-10a92056dd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:54.818849 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad415063-36b0-4d72-ba4a-10a92056dd17 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:54.819126 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad415063-36b0-4d72-ba4a-10a92056dd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:54.819397 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad415063-36b0-4d72-ba4a-10a92056dd17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:54.820127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad415063-36b0-4d72-ba4a-10a92056dd17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:54.821718 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad415063-36b0-4d72-ba4a-10a92056dd17 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:54.833334 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1223/2472] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:27:54.837922 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e2a23-8107-4acf-9393-3c585c3cc5bc admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:54.848581 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e2a23-8107-4acf-9393-3c585c3cc5bc admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:54.854028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187060b9-ead3-4885-80b6-59a32a979130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:54.854776 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187060b9-ead3-4885-80b6-59a32a979130 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:54.855133 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187060b9-ead3-4885-80b6-59a32a979130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:54.855480 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187060b9-ead3-4885-80b6-59a32a979130 admin admin] PATH_INFO: `/v3/projects/27e74b6a9b5e4394ae45a2e708319326/users/2a5b0c3a5e4843bba3a5da8ce25b4529/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:54.857435 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187060b9-ead3-4885-80b6-59a32a979130 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27e74b6a9b5e4394ae45a2e708319326, user_id=2a5b0c3a5e4843bba3a5da8ce25b4529, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:54.860127 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187060b9-ead3-4885-80b6-59a32a979130 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:54.871230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1250/2473] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:27:54 2022] PUT /identity/v3/projects/27e74b6a9b5e4394ae45a2e708319326/users/2a5b0c3a5e4843bba3a5da8ce25b4529/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:27:54.882864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04cb7d0-a14e-46c0-b9c8-8570724020fb None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:54.883151 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04cb7d0-a14e-46c0-b9c8-8570724020fb None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:54.883425 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04cb7d0-a14e-46c0-b9c8-8570724020fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:54.913072 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-a04cb7d0-a14e-46c0-b9c8-8570724020fb None None] MFA Rules not processed for user `2a5b0c3a5e4843bba3a5da8ce25b4529`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:27:54.914876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-a04cb7d0-a14e-46c0-b9c8-8570724020fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:27:54.986814 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1224/2474] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:27:54 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:27:54.997980 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187060b9-ead3-4885-80b6-59a32a979130 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:54.999824 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-187060b9-ead3-4885-80b6-59a32a979130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:27:55.095009 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187060b9-ead3-4885-80b6-59a32a979130 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:55.099496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5ab21-525a-4182-b455-ac2b6af07a6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:55.100519 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5ab21-525a-4182-b455-ac2b6af07a6c service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:55.100519 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5ab21-525a-4182-b455-ac2b6af07a6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:55.100749 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5ab21-525a-4182-b455-ac2b6af07a6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:55.109986 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b5ab21-525a-4182-b455-ac2b6af07a6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:55.111410 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b5ab21-525a-4182-b455-ac2b6af07a6c service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:55.121521 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1251/2475] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:27:54 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:27:55.204095 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04cb7d0-a14e-46c0-b9c8-8570724020fb None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:55.214945 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04cb7d0-a14e-46c0-b9c8-8570724020fb None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:55.218453 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd01f42-0b9f-45ea-b85c-466eeaac9dff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:55.219051 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd01f42-0b9f-45ea-b85c-466eeaac9dff service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:55.219326 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd01f42-0b9f-45ea-b85c-466eeaac9dff service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:55.219631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd01f42-0b9f-45ea-b85c-466eeaac9dff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:55.224819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:27:55.284754 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd01f42-0b9f-45ea-b85c-466eeaac9dff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:55.286247 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd01f42-0b9f-45ea-b85c-466eeaac9dff service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:55.295924 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1225/2476] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:27:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:28:05.116725 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=74,keystone:INSERT=5 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:05.292030 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=24 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:06.721759 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5ab21-525a-4182-b455-ac2b6af07a6c service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:06.784951 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5ab21-525a-4182-b455-ac2b6af07a6c service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:06.789563 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b671e9-c558-46c7-a927-322ec4e75a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:06.790190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b671e9-c558-46c7-a927-322ec4e75a9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:06.790469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b671e9-c558-46c7-a927-322ec4e75a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:06.790741 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b671e9-c558-46c7-a927-322ec4e75a9b admin admin] PATH_INFO: `/v3/users/899cdca5b777419ab86c68ff76aeac83` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:06.791883 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b671e9-c558-46c7-a927-322ec4e75a9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=899cdca5b777419ab86c68ff76aeac83)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:06.793829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b671e9-c558-46c7-a927-322ec4e75a9b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:06.846632 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e2b671e9-c558-46c7-a927-322ec4e75a9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '899cdca5b777419ab86c68ff76aeac83'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:06.853655 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e2b671e9-c558-46c7-a927-322ec4e75a9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '899cdca5b777419ab86c68ff76aeac83'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:06.854341 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e2b671e9-c558-46c7-a927-322ec4e75a9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '899cdca5b777419ab86c68ff76aeac83'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:06.854977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e2b671e9-c558-46c7-a927-322ec4e75a9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '899cdca5b777419ab86c68ff76aeac83'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:06.866597 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e2b671e9-c558-46c7-a927-322ec4e75a9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '899cdca5b777419ab86c68ff76aeac83'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:06.880302 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1252/2477] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:28:06 2022] DELETE /identity/v3/users/899cdca5b777419ab86c68ff76aeac83 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:06.885635 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33c45e-5eb4-443c-833e-9b5ceaf30c77 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:06.886157 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33c45e-5eb4-443c-833e-9b5ceaf30c77 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:06.886157 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33c45e-5eb4-443c-833e-9b5ceaf30c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:06.905606 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-bf33c45e-5eb4-443c-833e-9b5ceaf30c77 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:28:06.906793 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-bf33c45e-5eb4-443c-833e-9b5ceaf30c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:06.965464 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1226/2478] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:28:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:28:06.974926 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b671e9-c558-46c7-a927-322ec4e75a9b admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:06.976852 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e2b671e9-c558-46c7-a927-322ec4e75a9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:07.038137 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b671e9-c558-46c7-a927-322ec4e75a9b admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:07.041799 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c51c4e-3d1b-45c3-92de-a81628b2a47a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:07.042405 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c51c4e-3d1b-45c3-92de-a81628b2a47a service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:07.042685 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c51c4e-3d1b-45c3-92de-a81628b2a47a service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:07.042953 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c51c4e-3d1b-45c3-92de-a81628b2a47a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:07.051809 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c51c4e-3d1b-45c3-92de-a81628b2a47a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:07.053225 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c51c4e-3d1b-45c3-92de-a81628b2a47a service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:07.062446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1253/2479] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:28:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:28:07.334118 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf33c45e-5eb4-443c-833e-9b5ceaf30c77 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:07.335816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-bf33c45e-5eb4-443c-833e-9b5ceaf30c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:07.353467 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf33c45e-5eb4-443c-833e-9b5ceaf30c77 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:07.358221 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e7f0f-318f-4897-84cf-7673c544ab86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:07.359027 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e7f0f-318f-4897-84cf-7673c544ab86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:07.359278 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e7f0f-318f-4897-84cf-7673c544ab86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:07.359721 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e7f0f-318f-4897-84cf-7673c544ab86 admin admin] PATH_INFO: `/v3/projects/35d8d6a3514b4760b7a4d891e3395f15` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:07.360867 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e7f0f-318f-4897-84cf-7673c544ab86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35d8d6a3514b4760b7a4d891e3395f15)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:07.362805 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e7f0f-318f-4897-84cf-7673c544ab86 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:07.415481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-417e7f0f-318f-4897-84cf-7673c544ab86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35d8d6a3514b4760b7a4d891e3395f15', 'request_id': 'req-417e7f0f-318f-4897-84cf-7673c544ab86'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:07.422886 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-417e7f0f-318f-4897-84cf-7673c544ab86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35d8d6a3514b4760b7a4d891e3395f15', 'request_id': 'req-417e7f0f-318f-4897-84cf-7673c544ab86'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:07.428567 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-417e7f0f-318f-4897-84cf-7673c544ab86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35d8d6a3514b4760b7a4d891e3395f15', 'request_id': 'req-417e7f0f-318f-4897-84cf-7673c544ab86'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:07.439787 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-417e7f0f-318f-4897-84cf-7673c544ab86 admin admin] The token cache is being invalidate because project 35d8d6a3514b4760b7a4d891e3395f15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:28:07.440069 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-417e7f0f-318f-4897-84cf-7673c544ab86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:07.440685 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-417e7f0f-318f-4897-84cf-7673c544ab86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:07.442185 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1227/2480] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:28:07 2022] DELETE /identity/v3/projects/35d8d6a3514b4760b7a4d891e3395f15 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:07.446132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c51c4e-3d1b-45c3-92de-a81628b2a47a service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:07.455707 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c51c4e-3d1b-45c3-92de-a81628b2a47a service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:07.473596 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:07.514500 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18acffc0-22b5-469a-bc4f-b6f126e7443d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:07.515130 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18acffc0-22b5-469a-bc4f-b6f126e7443d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:07.515410 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18acffc0-22b5-469a-bc4f-b6f126e7443d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:07.515716 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18acffc0-22b5-469a-bc4f-b6f126e7443d admin admin] PATH_INFO: `/v3/users/2a5b0c3a5e4843bba3a5da8ce25b4529` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:07.516827 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18acffc0-22b5-469a-bc4f-b6f126e7443d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a5b0c3a5e4843bba3a5da8ce25b4529)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:07.518736 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18acffc0-22b5-469a-bc4f-b6f126e7443d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:07.566428 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-18acffc0-22b5-469a-bc4f-b6f126e7443d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a5b0c3a5e4843bba3a5da8ce25b4529'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:07.573411 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-18acffc0-22b5-469a-bc4f-b6f126e7443d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a5b0c3a5e4843bba3a5da8ce25b4529'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:07.574177 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-18acffc0-22b5-469a-bc4f-b6f126e7443d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a5b0c3a5e4843bba3a5da8ce25b4529'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:07.574761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-18acffc0-22b5-469a-bc4f-b6f126e7443d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a5b0c3a5e4843bba3a5da8ce25b4529'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:07.587140 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-18acffc0-22b5-469a-bc4f-b6f126e7443d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a5b0c3a5e4843bba3a5da8ce25b4529'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:07.599416 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1254/2481] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:28:07 2022] DELETE /identity/v3/users/2a5b0c3a5e4843bba3a5da8ce25b4529 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:07.872259 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e7f0f-318f-4897-84cf-7673c544ab86 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:07.873746 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-417e7f0f-318f-4897-84cf-7673c544ab86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:07.887431 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e7f0f-318f-4897-84cf-7673c544ab86 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:07.946496 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c3e9ab-b4c2-4fe7-bff7-c8cb51ad7c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:07.947121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3e9ab-b4c2-4fe7-bff7-c8cb51ad7c7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:07.947397 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3e9ab-b4c2-4fe7-bff7-c8cb51ad7c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:07.947704 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3e9ab-b4c2-4fe7-bff7-c8cb51ad7c7f admin admin] PATH_INFO: `/v3/projects/27e74b6a9b5e4394ae45a2e708319326` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:07.948810 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c3e9ab-b4c2-4fe7-bff7-c8cb51ad7c7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=27e74b6a9b5e4394ae45a2e708319326)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:07.950727 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c3e9ab-b4c2-4fe7-bff7-c8cb51ad7c7f admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:07.980732 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:07.998713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-71c3e9ab-b4c2-4fe7-bff7-c8cb51ad7c7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27e74b6a9b5e4394ae45a2e708319326', 'request_id': 'req-71c3e9ab-b4c2-4fe7-bff7-c8cb51ad7c7f'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:08.006394 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-71c3e9ab-b4c2-4fe7-bff7-c8cb51ad7c7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27e74b6a9b5e4394ae45a2e708319326', 'request_id': 'req-71c3e9ab-b4c2-4fe7-bff7-c8cb51ad7c7f'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:08.012938 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-71c3e9ab-b4c2-4fe7-bff7-c8cb51ad7c7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27e74b6a9b5e4394ae45a2e708319326', 'request_id': 'req-71c3e9ab-b4c2-4fe7-bff7-c8cb51ad7c7f'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:08.024668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-71c3e9ab-b4c2-4fe7-bff7-c8cb51ad7c7f admin admin] The token cache is being invalidate because project 27e74b6a9b5e4394ae45a2e708319326 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:28:08.024937 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-71c3e9ab-b4c2-4fe7-bff7-c8cb51ad7c7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:08.025631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-71c3e9ab-b4c2-4fe7-bff7-c8cb51ad7c7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:08.027209 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1228/2482] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:28:07 2022] DELETE /identity/v3/projects/27e74b6a9b5e4394ae45a2e708319326 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:08.032367 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59aa4f9-cd22-44b6-b39a-738fab3e1528 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:08.032861 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59aa4f9-cd22-44b6-b39a-738fab3e1528 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:08.032958 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59aa4f9-cd22-44b6-b39a-738fab3e1528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:08.052010 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-f59aa4f9-cd22-44b6-b39a-738fab3e1528 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:28:08.053203 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f59aa4f9-cd22-44b6-b39a-738fab3e1528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:08.111822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1255/2483] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:28:08.120526 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29377d-2580-4a4f-9459-0eb85413f14a None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:08.120790 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29377d-2580-4a4f-9459-0eb85413f14a None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:08.121108 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29377d-2580-4a4f-9459-0eb85413f14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:08.141787 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-4b29377d-2580-4a4f-9459-0eb85413f14a None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:28:08.143007 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-4b29377d-2580-4a4f-9459-0eb85413f14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:08.150879 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1229/2484] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:28:08.155300 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59aa4f9-cd22-44b6-b39a-738fab3e1528 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:08.164761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59aa4f9-cd22-44b6-b39a-738fab3e1528 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:08.169334 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c4c14-0f56-479d-9c3f-bde59578059b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:08.169925 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c4c14-0f56-479d-9c3f-bde59578059b admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:08.170200 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c4c14-0f56-479d-9c3f-bde59578059b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:08.170470 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c4c14-0f56-479d-9c3f-bde59578059b admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:08.171108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c4c14-0f56-479d-9c3f-bde59578059b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:28:08.171407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c4c14-0f56-479d-9c3f-bde59578059b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:08.173085 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c4c14-0f56-479d-9c3f-bde59578059b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:08.184774 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1256/2485] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:28:08.189127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29377d-2580-4a4f-9459-0eb85413f14a None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:08.198966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29377d-2580-4a4f-9459-0eb85413f14a None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:08.204406 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af2b32-a64a-49e3-82b1-a47fd5cdc1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:08.205012 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af2b32-a64a-49e3-82b1-a47fd5cdc1bf admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:08.205294 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af2b32-a64a-49e3-82b1-a47fd5cdc1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:08.205569 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af2b32-a64a-49e3-82b1-a47fd5cdc1bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:08.206209 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af2b32-a64a-49e3-82b1-a47fd5cdc1bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:08.208173 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af2b32-a64a-49e3-82b1-a47fd5cdc1bf admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:08.236383 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1230/2486] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:28:08 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:28:08.240569 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c4c14-0f56-479d-9c3f-bde59578059b admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:08.251049 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c4c14-0f56-479d-9c3f-bde59578059b admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:08.276342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:08.309377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7a8516-6b73-4d91-90cc-705a6f964d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:08.309978 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a8516-6b73-4d91-90cc-705a6f964d18 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:08.310257 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a8516-6b73-4d91-90cc-705a6f964d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:08.310528 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a8516-6b73-4d91-90cc-705a6f964d18 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:08.311165 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7a8516-6b73-4d91-90cc-705a6f964d18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:08.313104 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7a8516-6b73-4d91-90cc-705a6f964d18 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:08.334204 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1257/2487] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:28:08 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:28:08.338503 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af2b32-a64a-49e3-82b1-a47fd5cdc1bf admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:08.348846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af2b32-a64a-49e3-82b1-a47fd5cdc1bf admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:08.353508 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb5f43-69a3-47c6-9467-d7b049d1fa28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:08.354089 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb5f43-69a3-47c6-9467-d7b049d1fa28 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:08.354361 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb5f43-69a3-47c6-9467-d7b049d1fa28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:08.354628 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb5f43-69a3-47c6-9467-d7b049d1fa28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:08.355260 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb5f43-69a3-47c6-9467-d7b049d1fa28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:08.356856 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb5f43-69a3-47c6-9467-d7b049d1fa28 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:08.366449 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1231/2488] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:28:08.371022 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7a8516-6b73-4d91-90cc-705a6f964d18 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:08.381448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7a8516-6b73-4d91-90cc-705a6f964d18 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:08.386747 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc4df1-5ba3-423f-952b-811f674fcc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:08.387338 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc4df1-5ba3-423f-952b-811f674fcc85 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:08.387649 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc4df1-5ba3-423f-952b-811f674fcc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:08.387929 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc4df1-5ba3-423f-952b-811f674fcc85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:08.388568 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdc4df1-5ba3-423f-952b-811f674fcc85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:08.390163 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdc4df1-5ba3-423f-952b-811f674fcc85 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:08.400054 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1258/2489] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:28:08.404467 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb5f43-69a3-47c6-9467-d7b049d1fa28 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:08.414406 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb5f43-69a3-47c6-9467-d7b049d1fa28 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:08.418959 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49591913-ea1d-47a8-b534-db443d32bd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:08.419623 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49591913-ea1d-47a8-b534-db443d32bd21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:08.419900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49591913-ea1d-47a8-b534-db443d32bd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:08.420170 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49591913-ea1d-47a8-b534-db443d32bd21 admin admin] PATH_INFO: `/v3/projects/9052a72bc9614f1082d7b7737bbd2377/users/2daebf1c4b0c44af935fc837eea91fcc/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:08.437959 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49591913-ea1d-47a8-b534-db443d32bd21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9052a72bc9614f1082d7b7737bbd2377, user_id=2daebf1c4b0c44af935fc837eea91fcc, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:08.441269 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49591913-ea1d-47a8-b534-db443d32bd21 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:08.455000 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1232/2490] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:28:08 2022] PUT /identity/v3/projects/9052a72bc9614f1082d7b7737bbd2377/users/2daebf1c4b0c44af935fc837eea91fcc/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:08.465269 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1adf01e-2d96-4bc1-8b23-e6434495b006 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:08.465560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1adf01e-2d96-4bc1-8b23-e6434495b006 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:08.465836 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1adf01e-2d96-4bc1-8b23-e6434495b006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:08.496014 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-c1adf01e-2d96-4bc1-8b23-e6434495b006 None None] MFA Rules not processed for user `2daebf1c4b0c44af935fc837eea91fcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:28:08.497250 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c1adf01e-2d96-4bc1-8b23-e6434495b006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:08.560123 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1259/2491] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:28:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:28:08.564619 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49591913-ea1d-47a8-b534-db443d32bd21 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:08.626551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49591913-ea1d-47a8-b534-db443d32bd21 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:08.631063 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61573408-f6f6-48cf-8afd-ba8d24b9612c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:08.631691 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61573408-f6f6-48cf-8afd-ba8d24b9612c admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:08.631965 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61573408-f6f6-48cf-8afd-ba8d24b9612c admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:08.632234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61573408-f6f6-48cf-8afd-ba8d24b9612c admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:08.632873 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61573408-f6f6-48cf-8afd-ba8d24b9612c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:08.634752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61573408-f6f6-48cf-8afd-ba8d24b9612c admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:08.661509 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1233/2492] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:28:08 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:28:08.666258 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1adf01e-2d96-4bc1-8b23-e6434495b006 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:08.677054 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1adf01e-2d96-4bc1-8b23-e6434495b006 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:08.733303 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1701d48-dd1f-460b-aac9-d20160567482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:08.733904 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1701d48-dd1f-460b-aac9-d20160567482 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:08.734179 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1701d48-dd1f-460b-aac9-d20160567482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:08.734452 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1701d48-dd1f-460b-aac9-d20160567482 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:08.735072 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1701d48-dd1f-460b-aac9-d20160567482 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:08.737015 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1701d48-dd1f-460b-aac9-d20160567482 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:08.758510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1260/2493] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:28:08 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:28:08.762637 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61573408-f6f6-48cf-8afd-ba8d24b9612c admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:08.772899 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61573408-f6f6-48cf-8afd-ba8d24b9612c admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:08.777649 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac38e02-fb9e-4b92-bf8c-ecbb5e07d881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:08.778509 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac38e02-fb9e-4b92-bf8c-ecbb5e07d881 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:08.778790 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac38e02-fb9e-4b92-bf8c-ecbb5e07d881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:08.779066 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac38e02-fb9e-4b92-bf8c-ecbb5e07d881 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:08.779746 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac38e02-fb9e-4b92-bf8c-ecbb5e07d881 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:08.781345 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac38e02-fb9e-4b92-bf8c-ecbb5e07d881 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:08.792543 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1234/2494] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:28:08.796715 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1701d48-dd1f-460b-aac9-d20160567482 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:08.807816 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1701d48-dd1f-460b-aac9-d20160567482 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:08.812535 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16328e23-26aa-45e5-a1de-081d4713d397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:08.813191 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16328e23-26aa-45e5-a1de-081d4713d397 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:08.813472 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16328e23-26aa-45e5-a1de-081d4713d397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:08.813781 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16328e23-26aa-45e5-a1de-081d4713d397 admin admin] PATH_INFO: `/v3/projects/8cd8064cdd9e4e6f9c513ecddbb2fb43/users/c40f28e7d0534d6c88f61e47ea405517/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:08.828504 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16328e23-26aa-45e5-a1de-081d4713d397 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cd8064cdd9e4e6f9c513ecddbb2fb43, user_id=c40f28e7d0534d6c88f61e47ea405517, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:08.831238 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16328e23-26aa-45e5-a1de-081d4713d397 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:08.844884 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1261/2495] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:28:08 2022] PUT /identity/v3/projects/8cd8064cdd9e4e6f9c513ecddbb2fb43/users/c40f28e7d0534d6c88f61e47ea405517/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:08.849000 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac38e02-fb9e-4b92-bf8c-ecbb5e07d881 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:08.859336 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac38e02-fb9e-4b92-bf8c-ecbb5e07d881 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:08.879073 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:08.917524 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f079da4-ac23-4625-b519-65b17e1a92ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:08.918195 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f079da4-ac23-4625-b519-65b17e1a92ae admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:08.918388 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f079da4-ac23-4625-b519-65b17e1a92ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:08.918660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f079da4-ac23-4625-b519-65b17e1a92ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:08.919305 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f079da4-ac23-4625-b519-65b17e1a92ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:08.920943 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f079da4-ac23-4625-b519-65b17e1a92ae admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:08.930595 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1235/2496] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:28:08.934912 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16328e23-26aa-45e5-a1de-081d4713d397 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:08.940296 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:08.947006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16328e23-26aa-45e5-a1de-081d4713d397 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:08.951770 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b30da-546f-493e-82b7-77fcb49c017c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:08.952424 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b30da-546f-493e-82b7-77fcb49c017c admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:08.952705 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b30da-546f-493e-82b7-77fcb49c017c admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:08.952982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b30da-546f-493e-82b7-77fcb49c017c admin admin] PATH_INFO: `/v3/projects/8cd8064cdd9e4e6f9c513ecddbb2fb43/users/c40f28e7d0534d6c88f61e47ea405517/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:08.954881 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938b30da-546f-493e-82b7-77fcb49c017c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cd8064cdd9e4e6f9c513ecddbb2fb43, user_id=c40f28e7d0534d6c88f61e47ea405517, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:08.957584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938b30da-546f-493e-82b7-77fcb49c017c admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:08.968209 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1262/2497] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:28:08 2022] PUT /identity/v3/projects/8cd8064cdd9e4e6f9c513ecddbb2fb43/users/c40f28e7d0534d6c88f61e47ea405517/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:08.980129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78381a-631c-477d-88ad-9acf2e9a7cae None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:08.980405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78381a-631c-477d-88ad-9acf2e9a7cae None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:08.980683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78381a-631c-477d-88ad-9acf2e9a7cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:09.011988 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-4c78381a-631c-477d-88ad-9acf2e9a7cae None None] MFA Rules not processed for user `c40f28e7d0534d6c88f61e47ea405517`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:28:09.013224 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-4c78381a-631c-477d-88ad-9acf2e9a7cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:09.088376 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1236/2498] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:28:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:28:09.103999 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b30da-546f-493e-82b7-77fcb49c017c admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:09.105810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-938b30da-546f-493e-82b7-77fcb49c017c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:09.166351 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b30da-546f-493e-82b7-77fcb49c017c admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:09.170367 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f7b67-ad50-489f-a52b-d1258f982881 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:09.171094 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f7b67-ad50-489f-a52b-d1258f982881 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:09.171500 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f7b67-ad50-489f-a52b-d1258f982881 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:09.171937 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f7b67-ad50-489f-a52b-d1258f982881 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:09.185500 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1f7b67-ad50-489f-a52b-d1258f982881 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:09.187869 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1f7b67-ad50-489f-a52b-d1258f982881 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:09.203232 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1263/2499] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:28:09 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:28:09.521919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78381a-631c-477d-88ad-9acf2e9a7cae None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:09.531919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78381a-631c-477d-88ad-9acf2e9a7cae None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:09.536482 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedb7e4-2f13-415c-a361-9da4c39f478f tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c40f28e7d0534d6c88f61e47ea405517', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8cd8064cdd9e4e6f9c513ecddbb2fb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:09.537078 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedb7e4-2f13-415c-a361-9da4c39f478f tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:09.537368 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedb7e4-2f13-415c-a361-9da4c39f478f tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:09.537645 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedb7e4-2f13-415c-a361-9da4c39f478f tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:09.538297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dedb7e4-2f13-415c-a361-9da4c39f478f tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:28:09.538605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dedb7e4-2f13-415c-a361-9da4c39f478f tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:09.540308 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dedb7e4-2f13-415c-a361-9da4c39f478f tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:09.552120 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1237/2500] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:28:09.556246 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f7b67-ad50-489f-a52b-d1258f982881 service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:09.565182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f7b67-ad50-489f-a52b-d1258f982881 service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:09.569654 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b98f6d-7335-4c39-9b1c-2929c88b7009 tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c40f28e7d0534d6c88f61e47ea405517', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8cd8064cdd9e4e6f9c513ecddbb2fb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:09.570722 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b98f6d-7335-4c39-9b1c-2929c88b7009 tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:09.570996 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b98f6d-7335-4c39-9b1c-2929c88b7009 tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:09.571266 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b98f6d-7335-4c39-9b1c-2929c88b7009 tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:09.571920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b98f6d-7335-4c39-9b1c-2929c88b7009 tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:09.573833 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b98f6d-7335-4c39-9b1c-2929c88b7009 tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:09.600190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1264/2501] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:28:09 2022] POST /identity/v3/projects => generated 353 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:28:09.604362 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedb7e4-2f13-415c-a361-9da4c39f478f tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:09.614284 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedb7e4-2f13-415c-a361-9da4c39f478f tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:09.669361 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedeee95-487f-4dbe-8a6a-57c89caf6a3a tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c40f28e7d0534d6c88f61e47ea405517', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8cd8064cdd9e4e6f9c513ecddbb2fb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:09.669953 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedeee95-487f-4dbe-8a6a-57c89caf6a3a tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:09.670234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedeee95-487f-4dbe-8a6a-57c89caf6a3a tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:09.670508 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedeee95-487f-4dbe-8a6a-57c89caf6a3a tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:09.671151 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedeee95-487f-4dbe-8a6a-57c89caf6a3a tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:28:09.671460 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedeee95-487f-4dbe-8a6a-57c89caf6a3a tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:09.673068 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedeee95-487f-4dbe-8a6a-57c89caf6a3a tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:09.685195 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1238/2502] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:28:09.722515 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b98f6d-7335-4c39-9b1c-2929c88b7009 tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:09.732414 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b98f6d-7335-4c39-9b1c-2929c88b7009 tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:09.736902 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df39588-9d4d-49b1-9361-f83e0aa7eb32 tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c40f28e7d0534d6c88f61e47ea405517', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8cd8064cdd9e4e6f9c513ecddbb2fb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:09.737513 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df39588-9d4d-49b1-9361-f83e0aa7eb32 tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:09.737791 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df39588-9d4d-49b1-9361-f83e0aa7eb32 tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:09.738062 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df39588-9d4d-49b1-9361-f83e0aa7eb32 tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] PATH_INFO: `/v3/projects/83343a16836d491c8126a49d2a5b556c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:09.739169 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df39588-9d4d-49b1-9361-f83e0aa7eb32 tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] RBAC: Authorizing `identity:delete_project(project_id=83343a16836d491c8126a49d2a5b556c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:09.741093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df39588-9d4d-49b1-9361-f83e0aa7eb32 tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:09.788013 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-3df39588-9d4d-49b1-9361-f83e0aa7eb32 tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83343a16836d491c8126a49d2a5b556c', 'request_id': 'req-3df39588-9d4d-49b1-9361-f83e0aa7eb32'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:09.793461 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-3df39588-9d4d-49b1-9361-f83e0aa7eb32 tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83343a16836d491c8126a49d2a5b556c', 'request_id': 'req-3df39588-9d4d-49b1-9361-f83e0aa7eb32'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:09.800516 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-3df39588-9d4d-49b1-9361-f83e0aa7eb32 tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83343a16836d491c8126a49d2a5b556c', 'request_id': 'req-3df39588-9d4d-49b1-9361-f83e0aa7eb32'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:09.814241 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-3df39588-9d4d-49b1-9361-f83e0aa7eb32 tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] The token cache is being invalidate because project 83343a16836d491c8126a49d2a5b556c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:28:09.814563 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-3df39588-9d4d-49b1-9361-f83e0aa7eb32 tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:09.815159 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-3df39588-9d4d-49b1-9361-f83e0aa7eb32 tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:09.816716 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1265/2503] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:28:09 2022] DELETE /identity/v3/projects/83343a16836d491c8126a49d2a5b556c => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:10.008233 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedeee95-487f-4dbe-8a6a-57c89caf6a3a tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:10.010110 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-aedeee95-487f-4dbe-8a6a-57c89caf6a3a tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:10.040318 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=100 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:10.076178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedeee95-487f-4dbe-8a6a-57c89caf6a3a tempest-VolumeQuotaClassesTest-218763880 tempest-VolumeQuotaClassesTest-218763880-project-admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:10.131425 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf13698-95cd-41bd-b078-721b60d8011c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:10.132070 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf13698-95cd-41bd-b078-721b60d8011c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:10.132346 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf13698-95cd-41bd-b078-721b60d8011c admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:10.132615 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf13698-95cd-41bd-b078-721b60d8011c admin admin] PATH_INFO: `/v3/users/2daebf1c4b0c44af935fc837eea91fcc` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:10.133766 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf13698-95cd-41bd-b078-721b60d8011c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2daebf1c4b0c44af935fc837eea91fcc)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:10.136109 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf13698-95cd-41bd-b078-721b60d8011c admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:10.182649 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-7bf13698-95cd-41bd-b078-721b60d8011c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2daebf1c4b0c44af935fc837eea91fcc'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:10.190100 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-7bf13698-95cd-41bd-b078-721b60d8011c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2daebf1c4b0c44af935fc837eea91fcc'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:10.190701 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-7bf13698-95cd-41bd-b078-721b60d8011c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2daebf1c4b0c44af935fc837eea91fcc'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:10.203773 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-7bf13698-95cd-41bd-b078-721b60d8011c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2daebf1c4b0c44af935fc837eea91fcc'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:10.211062 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-7bf13698-95cd-41bd-b078-721b60d8011c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2daebf1c4b0c44af935fc837eea91fcc'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:10.217091 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1239/2504] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:28:10 2022] DELETE /identity/v3/users/2daebf1c4b0c44af935fc837eea91fcc => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:28:10.221484 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58de273-4c68-4c9c-9236-e4fe86ca226e None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:10.221977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58de273-4c68-4c9c-9236-e4fe86ca226e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:10.222085 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58de273-4c68-4c9c-9236-e4fe86ca226e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:10.240860 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-d58de273-4c68-4c9c-9236-e4fe86ca226e None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:28:10.246893 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d58de273-4c68-4c9c-9236-e4fe86ca226e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:10.288770 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:10.302442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1266/2505] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:28:10.312793 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf13698-95cd-41bd-b078-721b60d8011c admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:10.314203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-7bf13698-95cd-41bd-b078-721b60d8011c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:10.372760 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf13698-95cd-41bd-b078-721b60d8011c admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:10.376639 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b45ed-4269-4776-9df5-cf6ff9ad9d4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:10.377263 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b45ed-4269-4776-9df5-cf6ff9ad9d4e service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:10.377540 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b45ed-4269-4776-9df5-cf6ff9ad9d4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:10.377810 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b45ed-4269-4776-9df5-cf6ff9ad9d4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:10.384917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:10.387377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6b45ed-4269-4776-9df5-cf6ff9ad9d4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:10.389294 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6b45ed-4269-4776-9df5-cf6ff9ad9d4e service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:10.400605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1240/2506] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:28:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:28:10.667042 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58de273-4c68-4c9c-9236-e4fe86ca226e None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:10.668455 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d58de273-4c68-4c9c-9236-e4fe86ca226e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:10.680495 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58de273-4c68-4c9c-9236-e4fe86ca226e None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:10.685030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d6a44-360a-4bb3-8124-f2bdaeb0b7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:10.685661 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d6a44-360a-4bb3-8124-f2bdaeb0b7f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:10.685935 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d6a44-360a-4bb3-8124-f2bdaeb0b7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:10.686201 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d6a44-360a-4bb3-8124-f2bdaeb0b7f5 admin admin] PATH_INFO: `/v3/projects/9052a72bc9614f1082d7b7737bbd2377` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:10.687761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d6a44-360a-4bb3-8124-f2bdaeb0b7f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9052a72bc9614f1082d7b7737bbd2377)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:10.689267 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d6a44-360a-4bb3-8124-f2bdaeb0b7f5 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:10.735087 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d54d6a44-360a-4bb3-8124-f2bdaeb0b7f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9052a72bc9614f1082d7b7737bbd2377', 'request_id': 'req-d54d6a44-360a-4bb3-8124-f2bdaeb0b7f5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:10.740456 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d54d6a44-360a-4bb3-8124-f2bdaeb0b7f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9052a72bc9614f1082d7b7737bbd2377', 'request_id': 'req-d54d6a44-360a-4bb3-8124-f2bdaeb0b7f5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:10.746774 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d54d6a44-360a-4bb3-8124-f2bdaeb0b7f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9052a72bc9614f1082d7b7737bbd2377', 'request_id': 'req-d54d6a44-360a-4bb3-8124-f2bdaeb0b7f5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:10.759132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d54d6a44-360a-4bb3-8124-f2bdaeb0b7f5 admin admin] The token cache is being invalidate because project 9052a72bc9614f1082d7b7737bbd2377 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:28:10.759415 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d54d6a44-360a-4bb3-8124-f2bdaeb0b7f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:10.760072 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d54d6a44-360a-4bb3-8124-f2bdaeb0b7f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:10.762508 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1267/2507] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:28:10 2022] DELETE /identity/v3/projects/9052a72bc9614f1082d7b7737bbd2377 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:10.766646 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b45ed-4269-4776-9df5-cf6ff9ad9d4e service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:10.777146 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b45ed-4269-4776-9df5-cf6ff9ad9d4e service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:10.835708 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1757ea2-66b4-44b7-a5e3-b52b90735036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:10.836985 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1757ea2-66b4-44b7-a5e3-b52b90735036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:10.837268 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1757ea2-66b4-44b7-a5e3-b52b90735036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:10.837546 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1757ea2-66b4-44b7-a5e3-b52b90735036 admin admin] PATH_INFO: `/v3/users/c40f28e7d0534d6c88f61e47ea405517` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:10.838672 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1757ea2-66b4-44b7-a5e3-b52b90735036 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c40f28e7d0534d6c88f61e47ea405517)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:10.840651 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1757ea2-66b4-44b7-a5e3-b52b90735036 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:10.886114 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b1757ea2-66b4-44b7-a5e3-b52b90735036 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c40f28e7d0534d6c88f61e47ea405517'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:10.893498 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b1757ea2-66b4-44b7-a5e3-b52b90735036 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c40f28e7d0534d6c88f61e47ea405517'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:10.894202 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b1757ea2-66b4-44b7-a5e3-b52b90735036 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c40f28e7d0534d6c88f61e47ea405517'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:10.908022 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b1757ea2-66b4-44b7-a5e3-b52b90735036 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c40f28e7d0534d6c88f61e47ea405517'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:10.914899 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b1757ea2-66b4-44b7-a5e3-b52b90735036 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c40f28e7d0534d6c88f61e47ea405517'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:10.920827 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1241/2508] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:28:10 2022] DELETE /identity/v3/users/c40f28e7d0534d6c88f61e47ea405517 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:11.194202 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d6a44-360a-4bb3-8124-f2bdaeb0b7f5 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:11.196295 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d54d6a44-360a-4bb3-8124-f2bdaeb0b7f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:11.216840 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d6a44-360a-4bb3-8124-f2bdaeb0b7f5 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:11.279464 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ce9a3-2a70-4489-851f-a50d7087b22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:11.280150 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ce9a3-2a70-4489-851f-a50d7087b22b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:11.280396 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ce9a3-2a70-4489-851f-a50d7087b22b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:11.280670 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ce9a3-2a70-4489-851f-a50d7087b22b admin admin] PATH_INFO: `/v3/projects/8cd8064cdd9e4e6f9c513ecddbb2fb43` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:11.281784 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277ce9a3-2a70-4489-851f-a50d7087b22b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cd8064cdd9e4e6f9c513ecddbb2fb43)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:11.283731 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277ce9a3-2a70-4489-851f-a50d7087b22b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:11.333488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-277ce9a3-2a70-4489-851f-a50d7087b22b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cd8064cdd9e4e6f9c513ecddbb2fb43', 'request_id': 'req-277ce9a3-2a70-4489-851f-a50d7087b22b'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:11.338977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-277ce9a3-2a70-4489-851f-a50d7087b22b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cd8064cdd9e4e6f9c513ecddbb2fb43', 'request_id': 'req-277ce9a3-2a70-4489-851f-a50d7087b22b'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:11.345121 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-277ce9a3-2a70-4489-851f-a50d7087b22b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cd8064cdd9e4e6f9c513ecddbb2fb43', 'request_id': 'req-277ce9a3-2a70-4489-851f-a50d7087b22b'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:11.357861 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-277ce9a3-2a70-4489-851f-a50d7087b22b admin admin] The token cache is being invalidate because project 8cd8064cdd9e4e6f9c513ecddbb2fb43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:28:11.358143 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-277ce9a3-2a70-4489-851f-a50d7087b22b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:11.358757 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-277ce9a3-2a70-4489-851f-a50d7087b22b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:11.360591 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1268/2509] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:28:11 2022] DELETE /identity/v3/projects/8cd8064cdd9e4e6f9c513ecddbb2fb43 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:11.365651 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb122b4-e9ae-4a6d-bd39-f8e8f5816a3d None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:11.366118 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb122b4-e9ae-4a6d-bd39-f8e8f5816a3d None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:11.366209 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb122b4-e9ae-4a6d-bd39-f8e8f5816a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:11.387157 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-0eb122b4-e9ae-4a6d-bd39-f8e8f5816a3d None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:28:11.388425 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-0eb122b4-e9ae-4a6d-bd39-f8e8f5816a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:11.448734 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1242/2510] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:28:11.456448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201237e3-bf1a-47fc-b9fa-8a825a984dca None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:11.456734 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201237e3-bf1a-47fc-b9fa-8a825a984dca None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:11.457027 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201237e3-bf1a-47fc-b9fa-8a825a984dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:11.471634 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:11.478089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-201237e3-bf1a-47fc-b9fa-8a825a984dca None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:28:11.479997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-201237e3-bf1a-47fc-b9fa-8a825a984dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:11.488462 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1269/2511] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:28:11.493456 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb122b4-e9ae-4a6d-bd39-f8e8f5816a3d None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:11.503147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb122b4-e9ae-4a6d-bd39-f8e8f5816a3d None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:11.507805 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6750923-9ab0-4635-84b7-0ae1fdbbcf4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:11.508397 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6750923-9ab0-4635-84b7-0ae1fdbbcf4f admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:11.508678 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6750923-9ab0-4635-84b7-0ae1fdbbcf4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:11.508954 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6750923-9ab0-4635-84b7-0ae1fdbbcf4f admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:11.509948 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6750923-9ab0-4635-84b7-0ae1fdbbcf4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:28:11.510254 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6750923-9ab0-4635-84b7-0ae1fdbbcf4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:11.511870 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6750923-9ab0-4635-84b7-0ae1fdbbcf4f admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:11.523951 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1243/2512] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:28:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:28:11.528655 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201237e3-bf1a-47fc-b9fa-8a825a984dca None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:11.537996 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201237e3-bf1a-47fc-b9fa-8a825a984dca None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:11.543158 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e865e557-9fd8-4177-8775-c9e4d7e8a7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:11.543784 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865e557-9fd8-4177-8775-c9e4d7e8a7d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:11.544063 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865e557-9fd8-4177-8775-c9e4d7e8a7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:11.544335 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865e557-9fd8-4177-8775-c9e4d7e8a7d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:11.544976 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e865e557-9fd8-4177-8775-c9e4d7e8a7d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:11.546874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e865e557-9fd8-4177-8775-c9e4d7e8a7d8 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:11.573922 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1270/2513] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:28:11 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:28:11.578119 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6750923-9ab0-4635-84b7-0ae1fdbbcf4f admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:11.583301 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:11.589478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6750923-9ab0-4635-84b7-0ae1fdbbcf4f admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:11.646889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc69fa8-f2d4-43a1-b2f1-840e01e95a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:11.647487 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc69fa8-f2d4-43a1-b2f1-840e01e95a6e admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:11.647816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc69fa8-f2d4-43a1-b2f1-840e01e95a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:11.648073 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc69fa8-f2d4-43a1-b2f1-840e01e95a6e admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:11.648722 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc69fa8-f2d4-43a1-b2f1-840e01e95a6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:11.650630 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc69fa8-f2d4-43a1-b2f1-840e01e95a6e admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:11.672336 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1244/2514] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:28:11 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:28:11.676488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e865e557-9fd8-4177-8775-c9e4d7e8a7d8 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:11.687420 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e865e557-9fd8-4177-8775-c9e4d7e8a7d8 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:11.692034 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb872ab-5ac8-4ec7-97e6-97251a290479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:11.692626 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb872ab-5ac8-4ec7-97e6-97251a290479 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:11.692907 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb872ab-5ac8-4ec7-97e6-97251a290479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:11.693183 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb872ab-5ac8-4ec7-97e6-97251a290479 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:11.693826 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb872ab-5ac8-4ec7-97e6-97251a290479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:11.695453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb872ab-5ac8-4ec7-97e6-97251a290479 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:11.705735 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1271/2515] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:28:11.709935 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc69fa8-f2d4-43a1-b2f1-840e01e95a6e admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:11.720443 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc69fa8-f2d4-43a1-b2f1-840e01e95a6e admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:11.725491 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d55073a-ca1a-4380-b0f3-5c79daea96da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:11.726086 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55073a-ca1a-4380-b0f3-5c79daea96da admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:11.727211 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55073a-ca1a-4380-b0f3-5c79daea96da admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:11.727211 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55073a-ca1a-4380-b0f3-5c79daea96da admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:11.727828 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d55073a-ca1a-4380-b0f3-5c79daea96da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:11.729422 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d55073a-ca1a-4380-b0f3-5c79daea96da admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:11.740234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1245/2516] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:28:11.744957 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb872ab-5ac8-4ec7-97e6-97251a290479 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:11.754864 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb872ab-5ac8-4ec7-97e6-97251a290479 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:11.759419 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244732ca-5a3f-4dff-86e0-027ca7687c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:11.760096 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244732ca-5a3f-4dff-86e0-027ca7687c4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:11.760374 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244732ca-5a3f-4dff-86e0-027ca7687c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:11.760649 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244732ca-5a3f-4dff-86e0-027ca7687c4d admin admin] PATH_INFO: `/v3/projects/2d9201abdc6543d7875a00dbccd89dad/users/e477a7bf0aea400ca18f04df1ed90590/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:11.774778 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244732ca-5a3f-4dff-86e0-027ca7687c4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d9201abdc6543d7875a00dbccd89dad, user_id=e477a7bf0aea400ca18f04df1ed90590, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:11.777520 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244732ca-5a3f-4dff-86e0-027ca7687c4d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:11.791362 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1272/2517] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:28:11 2022] PUT /identity/v3/projects/2d9201abdc6543d7875a00dbccd89dad/users/e477a7bf0aea400ca18f04df1ed90590/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:11.803200 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba8ab54-e40d-4cf1-bfcd-5d01616965db None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:11.803483 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba8ab54-e40d-4cf1-bfcd-5d01616965db None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:11.803799 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba8ab54-e40d-4cf1-bfcd-5d01616965db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:11.832674 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-dba8ab54-e40d-4cf1-bfcd-5d01616965db None None] MFA Rules not processed for user `e477a7bf0aea400ca18f04df1ed90590`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:28:11.833890 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-dba8ab54-e40d-4cf1-bfcd-5d01616965db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:11.897725 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1246/2518] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:28:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:28:11.902758 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244732ca-5a3f-4dff-86e0-027ca7687c4d admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:11.969736 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244732ca-5a3f-4dff-86e0-027ca7687c4d admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:11.974364 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55f77f-4cad-4b37-b95d-f633dc748f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:11.974962 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55f77f-4cad-4b37-b95d-f633dc748f1a admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:11.975239 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55f77f-4cad-4b37-b95d-f633dc748f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:11.975509 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55f77f-4cad-4b37-b95d-f633dc748f1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:11.976197 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b55f77f-4cad-4b37-b95d-f633dc748f1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:11.978724 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b55f77f-4cad-4b37-b95d-f633dc748f1a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:12.006056 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1273/2519] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:28:11 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:28:12.010225 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba8ab54-e40d-4cf1-bfcd-5d01616965db None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:12.020990 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba8ab54-e40d-4cf1-bfcd-5d01616965db None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:12.076092 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10b43eb-73c3-43f4-bb51-ab7b74f8db0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:12.076622 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b43eb-73c3-43f4-bb51-ab7b74f8db0f admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:12.076901 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b43eb-73c3-43f4-bb51-ab7b74f8db0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:12.077170 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b43eb-73c3-43f4-bb51-ab7b74f8db0f admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:12.077789 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10b43eb-73c3-43f4-bb51-ab7b74f8db0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:12.079936 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10b43eb-73c3-43f4-bb51-ab7b74f8db0f admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:12.096477 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:12.104944 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1247/2520] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:28:12 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:28:12.106147 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55f77f-4cad-4b37-b95d-f633dc748f1a admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:12.116528 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55f77f-4cad-4b37-b95d-f633dc748f1a admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:12.121212 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f2942-4f89-4aa4-85e5-803c66f24082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:12.121807 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f2942-4f89-4aa4-85e5-803c66f24082 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:12.122089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f2942-4f89-4aa4-85e5-803c66f24082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:12.122365 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f2942-4f89-4aa4-85e5-803c66f24082 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:12.122998 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f2942-4f89-4aa4-85e5-803c66f24082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:12.124631 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f2942-4f89-4aa4-85e5-803c66f24082 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:12.135928 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1274/2521] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:28:12.139800 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10b43eb-73c3-43f4-bb51-ab7b74f8db0f admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:12.150204 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10b43eb-73c3-43f4-bb51-ab7b74f8db0f admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:12.155111 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d95e6-dea7-4045-a93d-ec965a71fe7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:12.155732 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d95e6-dea7-4045-a93d-ec965a71fe7c admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:12.156006 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d95e6-dea7-4045-a93d-ec965a71fe7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:12.156275 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d95e6-dea7-4045-a93d-ec965a71fe7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:12.156906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d95e6-dea7-4045-a93d-ec965a71fe7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:12.158472 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d95e6-dea7-4045-a93d-ec965a71fe7c admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:12.168725 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1248/2522] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:28:12.172763 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f2942-4f89-4aa4-85e5-803c66f24082 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:12.182636 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f2942-4f89-4aa4-85e5-803c66f24082 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:12.187257 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb42b61-33dd-48b7-85fa-1eae4d0596cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:12.187934 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb42b61-33dd-48b7-85fa-1eae4d0596cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:12.188215 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb42b61-33dd-48b7-85fa-1eae4d0596cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:12.188488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb42b61-33dd-48b7-85fa-1eae4d0596cf admin admin] PATH_INFO: `/v3/projects/b59af088635d4d17b487a82f5f383999/users/f8d0461a6e2b43fa93ba50eacec05458/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:12.203268 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb42b61-33dd-48b7-85fa-1eae4d0596cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b59af088635d4d17b487a82f5f383999, user_id=f8d0461a6e2b43fa93ba50eacec05458, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:12.205984 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb42b61-33dd-48b7-85fa-1eae4d0596cf admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:12.219631 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1275/2523] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:28:12 2022] PUT /identity/v3/projects/b59af088635d4d17b487a82f5f383999/users/f8d0461a6e2b43fa93ba50eacec05458/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:12.230383 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ea35a-c246-4987-9e4e-6e122d3ce102 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:12.230665 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ea35a-c246-4987-9e4e-6e122d3ce102 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:12.230938 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ea35a-c246-4987-9e4e-6e122d3ce102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:12.259852 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-490ea35a-c246-4987-9e4e-6e122d3ce102 None None] MFA Rules not processed for user `f8d0461a6e2b43fa93ba50eacec05458`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:28:12.261065 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-490ea35a-c246-4987-9e4e-6e122d3ce102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:12.325902 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1249/2524] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:28:12 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:28:12.330429 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb42b61-33dd-48b7-85fa-1eae4d0596cf admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:12.385223 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:12.397285 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb42b61-33dd-48b7-85fa-1eae4d0596cf admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:12.402621 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e61c6-9128-4ed0-ac6c-1a106824c7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:12.403476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e61c6-9128-4ed0-ac6c-1a106824c7da admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:12.403783 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e61c6-9128-4ed0-ac6c-1a106824c7da admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:12.404058 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e61c6-9128-4ed0-ac6c-1a106824c7da admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:12.404686 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e61c6-9128-4ed0-ac6c-1a106824c7da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:12.406580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e61c6-9128-4ed0-ac6c-1a106824c7da admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:12.433269 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1276/2525] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:28:12 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:28:12.437367 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ea35a-c246-4987-9e4e-6e122d3ce102 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:12.448440 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ea35a-c246-4987-9e4e-6e122d3ce102 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:12.507537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948c189-1cb9-49cb-b512-4237e3f7356e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:12.508692 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948c189-1cb9-49cb-b512-4237e3f7356e admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:12.508971 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948c189-1cb9-49cb-b512-4237e3f7356e admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:12.509243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948c189-1cb9-49cb-b512-4237e3f7356e admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:12.509914 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948c189-1cb9-49cb-b512-4237e3f7356e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:12.511879 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948c189-1cb9-49cb-b512-4237e3f7356e admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:12.534316 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1250/2526] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:28:12 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:28:12.538215 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e61c6-9128-4ed0-ac6c-1a106824c7da admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:12.548549 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e61c6-9128-4ed0-ac6c-1a106824c7da admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:12.553562 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202614f-b29b-44be-bd01-3b299043f558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:12.554159 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202614f-b29b-44be-bd01-3b299043f558 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:12.554457 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202614f-b29b-44be-bd01-3b299043f558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:12.554731 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202614f-b29b-44be-bd01-3b299043f558 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:12.555360 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202614f-b29b-44be-bd01-3b299043f558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:12.557006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202614f-b29b-44be-bd01-3b299043f558 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:12.567556 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1277/2527] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:28:12.572559 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948c189-1cb9-49cb-b512-4237e3f7356e admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:12.582925 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948c189-1cb9-49cb-b512-4237e3f7356e admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:12.588142 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa40b3a-a562-417d-a4d0-ce5a648540da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:12.588798 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa40b3a-a562-417d-a4d0-ce5a648540da admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:12.589079 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa40b3a-a562-417d-a4d0-ce5a648540da admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:12.589353 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa40b3a-a562-417d-a4d0-ce5a648540da admin admin] PATH_INFO: `/v3/projects/b13adca38240432993c3daca9e6a7dd1/users/744fb4b433eb4f46b7e2a34bca91d2ba/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:12.604562 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa40b3a-a562-417d-a4d0-ce5a648540da admin admin] RBAC: Authorizing `identity:create_grant(project_id=b13adca38240432993c3daca9e6a7dd1, user_id=744fb4b433eb4f46b7e2a34bca91d2ba, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:12.607202 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa40b3a-a562-417d-a4d0-ce5a648540da admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:12.620065 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1251/2528] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:28:12 2022] PUT /identity/v3/projects/b13adca38240432993c3daca9e6a7dd1/users/744fb4b433eb4f46b7e2a34bca91d2ba/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:12.624035 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202614f-b29b-44be-bd01-3b299043f558 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:12.634966 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202614f-b29b-44be-bd01-3b299043f558 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:12.689834 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a287d4f-7f80-4d40-bf6a-936328c25b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:12.690435 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a287d4f-7f80-4d40-bf6a-936328c25b06 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:12.690709 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a287d4f-7f80-4d40-bf6a-936328c25b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:12.690979 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a287d4f-7f80-4d40-bf6a-936328c25b06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:12.691634 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a287d4f-7f80-4d40-bf6a-936328c25b06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:12.693226 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a287d4f-7f80-4d40-bf6a-936328c25b06 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:12.703294 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1278/2529] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:28:12.707511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa40b3a-a562-417d-a4d0-ce5a648540da admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:12.717774 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa40b3a-a562-417d-a4d0-ce5a648540da admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:12.722297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143435a2-3a1d-400a-ac7f-f3a8aea0694a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:12.722935 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143435a2-3a1d-400a-ac7f-f3a8aea0694a admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:12.723209 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143435a2-3a1d-400a-ac7f-f3a8aea0694a admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:12.723478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143435a2-3a1d-400a-ac7f-f3a8aea0694a admin admin] PATH_INFO: `/v3/projects/b13adca38240432993c3daca9e6a7dd1/users/744fb4b433eb4f46b7e2a34bca91d2ba/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:12.725368 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143435a2-3a1d-400a-ac7f-f3a8aea0694a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b13adca38240432993c3daca9e6a7dd1, user_id=744fb4b433eb4f46b7e2a34bca91d2ba, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:12.728026 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143435a2-3a1d-400a-ac7f-f3a8aea0694a admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:12.738060 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1252/2530] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:28:12 2022] PUT /identity/v3/projects/b13adca38240432993c3daca9e6a7dd1/users/744fb4b433eb4f46b7e2a34bca91d2ba/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:12.748654 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087af92-a165-4e7b-94ce-b3986d9bf187 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:12.749011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087af92-a165-4e7b-94ce-b3986d9bf187 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:12.749394 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087af92-a165-4e7b-94ce-b3986d9bf187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:12.777822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-1087af92-a165-4e7b-94ce-b3986d9bf187 None None] MFA Rules not processed for user `744fb4b433eb4f46b7e2a34bca91d2ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:28:12.779298 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1087af92-a165-4e7b-94ce-b3986d9bf187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:12.846523 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1279/2531] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:28:12 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:28:12.857016 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143435a2-3a1d-400a-ac7f-f3a8aea0694a admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:12.858813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-143435a2-3a1d-400a-ac7f-f3a8aea0694a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:12.895203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:12.920336 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143435a2-3a1d-400a-ac7f-f3a8aea0694a admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:12.924035 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4151dc8-8d5f-43a2-9c2e-ae7e34d20d4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:12.924629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4151dc8-8d5f-43a2-9c2e-ae7e34d20d4c service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:12.924905 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4151dc8-8d5f-43a2-9c2e-ae7e34d20d4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:12.925178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4151dc8-8d5f-43a2-9c2e-ae7e34d20d4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:12.934630 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4151dc8-8d5f-43a2-9c2e-ae7e34d20d4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:12.936096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4151dc8-8d5f-43a2-9c2e-ae7e34d20d4c service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:12.946072 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1253/2532] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:28:12 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:28:13.441128 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087af92-a165-4e7b-94ce-b3986d9bf187 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:13.442579 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1087af92-a165-4e7b-94ce-b3986d9bf187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:13.454568 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087af92-a165-4e7b-94ce-b3986d9bf187 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:13.458172 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecaea8f-3c49-47b5-a89e-c103dc315f87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:13.458813 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecaea8f-3c49-47b5-a89e-c103dc315f87 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:13.459094 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecaea8f-3c49-47b5-a89e-c103dc315f87 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:13.459369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecaea8f-3c49-47b5-a89e-c103dc315f87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:13.504913 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:13.517759 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecaea8f-3c49-47b5-a89e-c103dc315f87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:13.519195 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecaea8f-3c49-47b5-a89e-c103dc315f87 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:13.528951 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1280/2533] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:28:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:28:17.564353 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4151dc8-8d5f-43a2-9c2e-ae7e34d20d4c service cinder] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:17.573519 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4151dc8-8d5f-43a2-9c2e-ae7e34d20d4c service cinder] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:17.577856 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee1202-b561-4fd1-8140-90c27ca1153d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:17.578439 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee1202-b561-4fd1-8140-90c27ca1153d service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:17.578718 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee1202-b561-4fd1-8140-90c27ca1153d service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:17.578990 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee1202-b561-4fd1-8140-90c27ca1153d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:17.635241 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ee1202-b561-4fd1-8140-90c27ca1153d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:17.636626 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ee1202-b561-4fd1-8140-90c27ca1153d service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:17.645324 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1254/2534] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:28:17 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:28:19.731383 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecaea8f-3c49-47b5-a89e-c103dc315f87 service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:19.798350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecaea8f-3c49-47b5-a89e-c103dc315f87 service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:19.803014 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8522ab2a-95ee-4ea6-85d3-601ba79fff2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:19.803688 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522ab2a-95ee-4ea6-85d3-601ba79fff2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:19.804019 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522ab2a-95ee-4ea6-85d3-601ba79fff2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:19.804298 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522ab2a-95ee-4ea6-85d3-601ba79fff2d admin admin] PATH_INFO: `/v3/users/e477a7bf0aea400ca18f04df1ed90590` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:19.805436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8522ab2a-95ee-4ea6-85d3-601ba79fff2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e477a7bf0aea400ca18f04df1ed90590)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:19.808014 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8522ab2a-95ee-4ea6-85d3-601ba79fff2d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:19.854553 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8522ab2a-95ee-4ea6-85d3-601ba79fff2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e477a7bf0aea400ca18f04df1ed90590'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:19.861404 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8522ab2a-95ee-4ea6-85d3-601ba79fff2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e477a7bf0aea400ca18f04df1ed90590'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:19.862067 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8522ab2a-95ee-4ea6-85d3-601ba79fff2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e477a7bf0aea400ca18f04df1ed90590'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:19.862681 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8522ab2a-95ee-4ea6-85d3-601ba79fff2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e477a7bf0aea400ca18f04df1ed90590'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:19.875137 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8522ab2a-95ee-4ea6-85d3-601ba79fff2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e477a7bf0aea400ca18f04df1ed90590'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:19.888392 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1281/2535] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:28:19 2022] DELETE /identity/v3/users/e477a7bf0aea400ca18f04df1ed90590 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:19.893024 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca741f9e-f0e9-4200-a45f-b8e753c7381a None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:19.893513 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca741f9e-f0e9-4200-a45f-b8e753c7381a None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:19.893609 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca741f9e-f0e9-4200-a45f-b8e753c7381a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:19.914200 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-ca741f9e-f0e9-4200-a45f-b8e753c7381a None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:28:19.915408 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-ca741f9e-f0e9-4200-a45f-b8e753c7381a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:19.974895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1255/2536] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:28:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:28:19.984838 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8522ab2a-95ee-4ea6-85d3-601ba79fff2d admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:19.986227 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8522ab2a-95ee-4ea6-85d3-601ba79fff2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:20.044178 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8522ab2a-95ee-4ea6-85d3-601ba79fff2d admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:20.047873 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41752c6e-6a8a-4108-af97-f3cafa2cb5b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:20.048480 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41752c6e-6a8a-4108-af97-f3cafa2cb5b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:20.048763 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41752c6e-6a8a-4108-af97-f3cafa2cb5b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:20.049037 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41752c6e-6a8a-4108-af97-f3cafa2cb5b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:20.058493 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41752c6e-6a8a-4108-af97-f3cafa2cb5b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:20.059973 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41752c6e-6a8a-4108-af97-f3cafa2cb5b4 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:20.070156 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1282/2537] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:28:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:28:20.339017 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca741f9e-f0e9-4200-a45f-b8e753c7381a None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:20.340535 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-ca741f9e-f0e9-4200-a45f-b8e753c7381a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:20.356371 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca741f9e-f0e9-4200-a45f-b8e753c7381a None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:20.361322 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18182c9-2f90-4cf6-8a65-4a936ed4b9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:20.361962 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18182c9-2f90-4cf6-8a65-4a936ed4b9ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:20.362272 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18182c9-2f90-4cf6-8a65-4a936ed4b9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:20.362544 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18182c9-2f90-4cf6-8a65-4a936ed4b9ef admin admin] PATH_INFO: `/v3/projects/2d9201abdc6543d7875a00dbccd89dad` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:20.363710 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18182c9-2f90-4cf6-8a65-4a936ed4b9ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d9201abdc6543d7875a00dbccd89dad)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:20.366633 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18182c9-2f90-4cf6-8a65-4a936ed4b9ef admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:20.420278 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b18182c9-2f90-4cf6-8a65-4a936ed4b9ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d9201abdc6543d7875a00dbccd89dad', 'request_id': 'req-b18182c9-2f90-4cf6-8a65-4a936ed4b9ef'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:20.428327 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b18182c9-2f90-4cf6-8a65-4a936ed4b9ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d9201abdc6543d7875a00dbccd89dad', 'request_id': 'req-b18182c9-2f90-4cf6-8a65-4a936ed4b9ef'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:20.434955 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b18182c9-2f90-4cf6-8a65-4a936ed4b9ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d9201abdc6543d7875a00dbccd89dad', 'request_id': 'req-b18182c9-2f90-4cf6-8a65-4a936ed4b9ef'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:20.436835 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:20.450490 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b18182c9-2f90-4cf6-8a65-4a936ed4b9ef admin admin] The token cache is being invalidate because project 2d9201abdc6543d7875a00dbccd89dad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:28:20.450490 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b18182c9-2f90-4cf6-8a65-4a936ed4b9ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:20.451346 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b18182c9-2f90-4cf6-8a65-4a936ed4b9ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:20.453424 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1256/2538] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:28:20 2022] DELETE /identity/v3/projects/2d9201abdc6543d7875a00dbccd89dad => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:20.457791 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41752c6e-6a8a-4108-af97-f3cafa2cb5b4 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:20.469982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41752c6e-6a8a-4108-af97-f3cafa2cb5b4 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:20.474696 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:20.528119 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f8ec8-ddc5-47ae-89f8-76e1e6ba0a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:20.528917 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f8ec8-ddc5-47ae-89f8-76e1e6ba0a27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:20.529192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f8ec8-ddc5-47ae-89f8-76e1e6ba0a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:20.529461 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f8ec8-ddc5-47ae-89f8-76e1e6ba0a27 admin admin] PATH_INFO: `/v3/users/f8d0461a6e2b43fa93ba50eacec05458` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:20.530561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306f8ec8-ddc5-47ae-89f8-76e1e6ba0a27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8d0461a6e2b43fa93ba50eacec05458)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:20.532500 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306f8ec8-ddc5-47ae-89f8-76e1e6ba0a27 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:20.578308 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-306f8ec8-ddc5-47ae-89f8-76e1e6ba0a27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8d0461a6e2b43fa93ba50eacec05458'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:20.584959 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-306f8ec8-ddc5-47ae-89f8-76e1e6ba0a27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8d0461a6e2b43fa93ba50eacec05458'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:20.585609 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-306f8ec8-ddc5-47ae-89f8-76e1e6ba0a27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8d0461a6e2b43fa93ba50eacec05458'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:20.586207 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-306f8ec8-ddc5-47ae-89f8-76e1e6ba0a27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8d0461a6e2b43fa93ba50eacec05458'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:20.597744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-306f8ec8-ddc5-47ae-89f8-76e1e6ba0a27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8d0461a6e2b43fa93ba50eacec05458'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:20.611294 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1283/2539] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:28:20 2022] DELETE /identity/v3/users/f8d0461a6e2b43fa93ba50eacec05458 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:20.883820 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18182c9-2f90-4cf6-8a65-4a936ed4b9ef admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:20.885310 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-b18182c9-2f90-4cf6-8a65-4a936ed4b9ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:20.898668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18182c9-2f90-4cf6-8a65-4a936ed4b9ef admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:20.965604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e4025-d419-41f9-8280-99eb083a5282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:20.966267 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e4025-d419-41f9-8280-99eb083a5282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:20.966578 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e4025-d419-41f9-8280-99eb083a5282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:20.966854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e4025-d419-41f9-8280-99eb083a5282 admin admin] PATH_INFO: `/v3/projects/b59af088635d4d17b487a82f5f383999` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:20.968092 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372e4025-d419-41f9-8280-99eb083a5282 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b59af088635d4d17b487a82f5f383999)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:20.970036 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372e4025-d419-41f9-8280-99eb083a5282 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:21.017253 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-372e4025-d419-41f9-8280-99eb083a5282 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b59af088635d4d17b487a82f5f383999', 'request_id': 'req-372e4025-d419-41f9-8280-99eb083a5282'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:21.024208 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-372e4025-d419-41f9-8280-99eb083a5282 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b59af088635d4d17b487a82f5f383999', 'request_id': 'req-372e4025-d419-41f9-8280-99eb083a5282'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:21.030760 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-372e4025-d419-41f9-8280-99eb083a5282 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b59af088635d4d17b487a82f5f383999', 'request_id': 'req-372e4025-d419-41f9-8280-99eb083a5282'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:21.042290 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-372e4025-d419-41f9-8280-99eb083a5282 admin admin] The token cache is being invalidate because project b59af088635d4d17b487a82f5f383999 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:28:21.042578 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-372e4025-d419-41f9-8280-99eb083a5282 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:21.043242 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-372e4025-d419-41f9-8280-99eb083a5282 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:21.044748 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1257/2540] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:28:20 2022] DELETE /identity/v3/projects/b59af088635d4d17b487a82f5f383999 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:21.048855 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f8ec8-ddc5-47ae-89f8-76e1e6ba0a27 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:21.050266 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-306f8ec8-ddc5-47ae-89f8-76e1e6ba0a27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:21.062764 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f8ec8-ddc5-47ae-89f8-76e1e6ba0a27 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:21.118290 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c3d3d-7855-4606-b975-cae35a29a8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:21.118916 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c3d3d-7855-4606-b975-cae35a29a8b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:21.119193 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c3d3d-7855-4606-b975-cae35a29a8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:21.119463 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c3d3d-7855-4606-b975-cae35a29a8b5 admin admin] PATH_INFO: `/v3/users/744fb4b433eb4f46b7e2a34bca91d2ba` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:21.120595 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c3d3d-7855-4606-b975-cae35a29a8b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=744fb4b433eb4f46b7e2a34bca91d2ba)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:21.122501 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c3d3d-7855-4606-b975-cae35a29a8b5 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:21.154647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:21.168898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a64c3d3d-7855-4606-b975-cae35a29a8b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '744fb4b433eb4f46b7e2a34bca91d2ba'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:21.175842 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a64c3d3d-7855-4606-b975-cae35a29a8b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '744fb4b433eb4f46b7e2a34bca91d2ba'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:21.176503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a64c3d3d-7855-4606-b975-cae35a29a8b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '744fb4b433eb4f46b7e2a34bca91d2ba'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:21.177097 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a64c3d3d-7855-4606-b975-cae35a29a8b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '744fb4b433eb4f46b7e2a34bca91d2ba'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:21.188840 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a64c3d3d-7855-4606-b975-cae35a29a8b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '744fb4b433eb4f46b7e2a34bca91d2ba'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:21.202006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1284/2541] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:28:21 2022] DELETE /identity/v3/users/744fb4b433eb4f46b7e2a34bca91d2ba => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:21.470408 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e4025-d419-41f9-8280-99eb083a5282 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:21.471907 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-372e4025-d419-41f9-8280-99eb083a5282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:21.486334 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e4025-d419-41f9-8280-99eb083a5282 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:21.545726 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c453781-fd0f-4d4f-9a00-e15657a09504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:21.546351 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c453781-fd0f-4d4f-9a00-e15657a09504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:21.546634 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c453781-fd0f-4d4f-9a00-e15657a09504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:21.546909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c453781-fd0f-4d4f-9a00-e15657a09504 admin admin] PATH_INFO: `/v3/projects/b13adca38240432993c3daca9e6a7dd1` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:21.548115 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c453781-fd0f-4d4f-9a00-e15657a09504 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b13adca38240432993c3daca9e6a7dd1)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:21.550004 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c453781-fd0f-4d4f-9a00-e15657a09504 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:21.581776 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:21.597307 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-1c453781-fd0f-4d4f-9a00-e15657a09504 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b13adca38240432993c3daca9e6a7dd1', 'request_id': 'req-1c453781-fd0f-4d4f-9a00-e15657a09504'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:21.605616 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-1c453781-fd0f-4d4f-9a00-e15657a09504 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b13adca38240432993c3daca9e6a7dd1', 'request_id': 'req-1c453781-fd0f-4d4f-9a00-e15657a09504'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:21.611268 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-1c453781-fd0f-4d4f-9a00-e15657a09504 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b13adca38240432993c3daca9e6a7dd1', 'request_id': 'req-1c453781-fd0f-4d4f-9a00-e15657a09504'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:21.622913 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-1c453781-fd0f-4d4f-9a00-e15657a09504 admin admin] The token cache is being invalidate because project b13adca38240432993c3daca9e6a7dd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:28:21.623199 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-1c453781-fd0f-4d4f-9a00-e15657a09504 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:21.623859 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-1c453781-fd0f-4d4f-9a00-e15657a09504 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:21.625364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1258/2542] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:28:21 2022] DELETE /identity/v3/projects/b13adca38240432993c3daca9e6a7dd1 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:21.631369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a070a-13a5-4c5c-b8b8-58a402bfc93d None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:21.631690 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a070a-13a5-4c5c-b8b8-58a402bfc93d None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:21.631969 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a070a-13a5-4c5c-b8b8-58a402bfc93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:21.651853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-770a070a-13a5-4c5c-b8b8-58a402bfc93d None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:28:21.653084 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-770a070a-13a5-4c5c-b8b8-58a402bfc93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:21.711254 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1285/2543] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:28:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:28:21.775449 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d78284-3752-4312-aded-dde90a6c8dc7 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:21.775785 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d78284-3752-4312-aded-dde90a6c8dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:21.776068 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d78284-3752-4312-aded-dde90a6c8dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:21.797076 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-26d78284-3752-4312-aded-dde90a6c8dc7 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:28:21.798346 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-26d78284-3752-4312-aded-dde90a6c8dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:21.806601 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1259/2544] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:28:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:28:21.811942 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a070a-13a5-4c5c-b8b8-58a402bfc93d None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:21.821809 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a070a-13a5-4c5c-b8b8-58a402bfc93d None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:21.827043 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94df86b-7304-4ba6-8207-cf2d2eb03177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:21.827679 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94df86b-7304-4ba6-8207-cf2d2eb03177 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:21.827962 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94df86b-7304-4ba6-8207-cf2d2eb03177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:21.828236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94df86b-7304-4ba6-8207-cf2d2eb03177 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:21.828907 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94df86b-7304-4ba6-8207-cf2d2eb03177 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:28:21.829210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94df86b-7304-4ba6-8207-cf2d2eb03177 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:21.830803 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94df86b-7304-4ba6-8207-cf2d2eb03177 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:21.842738 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1286/2545] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:28:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:28:21.847060 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d78284-3752-4312-aded-dde90a6c8dc7 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:21.857135 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d78284-3752-4312-aded-dde90a6c8dc7 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:21.861825 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f89b8-e277-4429-a5e5-03383ff4b65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:21.862411 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f89b8-e277-4429-a5e5-03383ff4b65b admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:21.862690 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f89b8-e277-4429-a5e5-03383ff4b65b admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:21.862963 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f89b8-e277-4429-a5e5-03383ff4b65b admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:21.863629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f89b8-e277-4429-a5e5-03383ff4b65b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:21.865533 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f89b8-e277-4429-a5e5-03383ff4b65b admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:21.892707 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1260/2546] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:28:21 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:28:21.896944 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94df86b-7304-4ba6-8207-cf2d2eb03177 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:21.908125 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94df86b-7304-4ba6-8207-cf2d2eb03177 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:21.963676 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df0876-3779-4b86-9c0b-3addf2d23d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:21.964290 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df0876-3779-4b86-9c0b-3addf2d23d17 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:21.964577 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df0876-3779-4b86-9c0b-3addf2d23d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:21.965482 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df0876-3779-4b86-9c0b-3addf2d23d17 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:21.966115 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df0876-3779-4b86-9c0b-3addf2d23d17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:21.968062 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df0876-3779-4b86-9c0b-3addf2d23d17 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:21.989684 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1287/2547] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:28:21 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:28:21.993918 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f89b8-e277-4429-a5e5-03383ff4b65b admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:22.004516 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f89b8-e277-4429-a5e5-03383ff4b65b admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:22.009205 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6775e-b45c-4eaf-95cd-887010beb55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:22.009796 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6775e-b45c-4eaf-95cd-887010beb55e admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:22.010074 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6775e-b45c-4eaf-95cd-887010beb55e admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:22.010343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6775e-b45c-4eaf-95cd-887010beb55e admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:22.011006 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c6775e-b45c-4eaf-95cd-887010beb55e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:22.013121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c6775e-b45c-4eaf-95cd-887010beb55e admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:22.023563 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1261/2548] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:28:22.028498 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df0876-3779-4b86-9c0b-3addf2d23d17 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:22.038477 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df0876-3779-4b86-9c0b-3addf2d23d17 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:22.043384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb536c5-90f1-4258-8b0d-0c6ab9bb8213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:22.044014 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb536c5-90f1-4258-8b0d-0c6ab9bb8213 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:22.044289 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb536c5-90f1-4258-8b0d-0c6ab9bb8213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:22.044583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb536c5-90f1-4258-8b0d-0c6ab9bb8213 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:22.045232 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb536c5-90f1-4258-8b0d-0c6ab9bb8213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:22.046799 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb536c5-90f1-4258-8b0d-0c6ab9bb8213 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:22.057422 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1288/2549] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:28:22.061427 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6775e-b45c-4eaf-95cd-887010beb55e admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:22.071287 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6775e-b45c-4eaf-95cd-887010beb55e admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:22.076454 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca92860-5097-4d3d-8dfd-3c0c7a4727aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:22.077089 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca92860-5097-4d3d-8dfd-3c0c7a4727aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:22.077371 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca92860-5097-4d3d-8dfd-3c0c7a4727aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:22.077648 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca92860-5097-4d3d-8dfd-3c0c7a4727aa admin admin] PATH_INFO: `/v3/projects/e54af3f9df6c43638e4cdf20df1d0711/users/78a7046d6de64d849abecd4c9f6ef5e2/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:22.092551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca92860-5097-4d3d-8dfd-3c0c7a4727aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=e54af3f9df6c43638e4cdf20df1d0711, user_id=78a7046d6de64d849abecd4c9f6ef5e2, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:22.095208 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca92860-5097-4d3d-8dfd-3c0c7a4727aa admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:22.108147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1262/2550] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:28:22 2022] PUT /identity/v3/projects/e54af3f9df6c43638e4cdf20df1d0711/users/78a7046d6de64d849abecd4c9f6ef5e2/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:22.119502 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf71eb4-6bb7-4c3f-9e03-960181ec2857 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:22.119863 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf71eb4-6bb7-4c3f-9e03-960181ec2857 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:22.120132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf71eb4-6bb7-4c3f-9e03-960181ec2857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:22.144917 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:22.148368 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-acf71eb4-6bb7-4c3f-9e03-960181ec2857 None None] MFA Rules not processed for user `78a7046d6de64d849abecd4c9f6ef5e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:28:22.150105 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-acf71eb4-6bb7-4c3f-9e03-960181ec2857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:22.214582 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1289/2551] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:28:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:28:22.219819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca92860-5097-4d3d-8dfd-3c0c7a4727aa admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:22.280748 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca92860-5097-4d3d-8dfd-3c0c7a4727aa admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:22.285429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1078559b-cdaf-4226-b0c7-16dea26699a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:22.286043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078559b-cdaf-4226-b0c7-16dea26699a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:22.286320 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078559b-cdaf-4226-b0c7-16dea26699a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:22.286588 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078559b-cdaf-4226-b0c7-16dea26699a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:22.287210 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1078559b-cdaf-4226-b0c7-16dea26699a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:22.289309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1078559b-cdaf-4226-b0c7-16dea26699a6 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:22.315991 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1263/2552] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:28:22 2022] POST /identity/v3/projects => generated 389 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:28:22.320048 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf71eb4-6bb7-4c3f-9e03-960181ec2857 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:22.331202 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf71eb4-6bb7-4c3f-9e03-960181ec2857 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:22.389616 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae68a4-5699-4ad2-b7bb-78799f97bfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:22.390222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae68a4-5699-4ad2-b7bb-78799f97bfc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:22.390498 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae68a4-5699-4ad2-b7bb-78799f97bfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:22.390797 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae68a4-5699-4ad2-b7bb-78799f97bfc5 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:22.391438 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae68a4-5699-4ad2-b7bb-78799f97bfc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:22.393369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae68a4-5699-4ad2-b7bb-78799f97bfc5 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:22.414549 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1290/2553] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:28:22 2022] POST /identity/v3/users => generated 301 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:28:22.418499 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1078559b-cdaf-4226-b0c7-16dea26699a6 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:22.428832 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1078559b-cdaf-4226-b0c7-16dea26699a6 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:22.433492 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc4d3a-99ca-4fa8-8a4a-ed1b8d06defa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:22.434572 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc4d3a-99ca-4fa8-8a4a-ed1b8d06defa admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:22.434853 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc4d3a-99ca-4fa8-8a4a-ed1b8d06defa admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:22.435157 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc4d3a-99ca-4fa8-8a4a-ed1b8d06defa admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:22.435859 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfc4d3a-99ca-4fa8-8a4a-ed1b8d06defa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:22.437456 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfc4d3a-99ca-4fa8-8a4a-ed1b8d06defa admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:22.448040 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1264/2554] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:28:22.453017 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae68a4-5699-4ad2-b7bb-78799f97bfc5 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:22.462932 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae68a4-5699-4ad2-b7bb-78799f97bfc5 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:22.467632 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e0061-a4a7-4ca4-82f4-ef9a424c4398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:22.468284 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e0061-a4a7-4ca4-82f4-ef9a424c4398 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:22.468604 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e0061-a4a7-4ca4-82f4-ef9a424c4398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:22.468839 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e0061-a4a7-4ca4-82f4-ef9a424c4398 admin admin] PATH_INFO: `/v3/projects/169d4b33178640ecb336eb9615951b4c/users/c35423ad984d43e4a75c77a48ed7b742/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:22.483660 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53e0061-a4a7-4ca4-82f4-ef9a424c4398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=169d4b33178640ecb336eb9615951b4c, user_id=c35423ad984d43e4a75c77a48ed7b742, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:22.486311 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53e0061-a4a7-4ca4-82f4-ef9a424c4398 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:22.499668 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1291/2555] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:28:22 2022] PUT /identity/v3/projects/169d4b33178640ecb336eb9615951b4c/users/c35423ad984d43e4a75c77a48ed7b742/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:22.503591 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc4d3a-99ca-4fa8-8a4a-ed1b8d06defa admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:22.514742 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc4d3a-99ca-4fa8-8a4a-ed1b8d06defa admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:22.539209 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:22.573290 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd743f-15fe-4d42-bbeb-a0f0b46f9e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:22.573815 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd743f-15fe-4d42-bbeb-a0f0b46f9e04 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:22.574096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd743f-15fe-4d42-bbeb-a0f0b46f9e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:22.574905 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd743f-15fe-4d42-bbeb-a0f0b46f9e04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:22.575567 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cd743f-15fe-4d42-bbeb-a0f0b46f9e04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:22.577250 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cd743f-15fe-4d42-bbeb-a0f0b46f9e04 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:22.587261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1265/2556] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:28:22.591135 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e0061-a4a7-4ca4-82f4-ef9a424c4398 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:22.601467 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e0061-a4a7-4ca4-82f4-ef9a424c4398 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:22.606046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83265e-ec75-4d0f-8df9-a00adea2eac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:22.606694 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83265e-ec75-4d0f-8df9-a00adea2eac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:22.606972 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83265e-ec75-4d0f-8df9-a00adea2eac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:22.607247 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83265e-ec75-4d0f-8df9-a00adea2eac8 admin admin] PATH_INFO: `/v3/projects/169d4b33178640ecb336eb9615951b4c/users/c35423ad984d43e4a75c77a48ed7b742/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:22.609214 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec83265e-ec75-4d0f-8df9-a00adea2eac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=169d4b33178640ecb336eb9615951b4c, user_id=c35423ad984d43e4a75c77a48ed7b742, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:22.611893 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec83265e-ec75-4d0f-8df9-a00adea2eac8 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:22.622508 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1292/2557] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:28:22 2022] PUT /identity/v3/projects/169d4b33178640ecb336eb9615951b4c/users/c35423ad984d43e4a75c77a48ed7b742/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:22.632899 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc2eba2-0228-4283-babc-e09f0e53b739 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:22.633191 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc2eba2-0228-4283-babc-e09f0e53b739 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:22.633469 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc2eba2-0228-4283-babc-e09f0e53b739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:22.661647 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-5fc2eba2-0228-4283-babc-e09f0e53b739 None None] MFA Rules not processed for user `c35423ad984d43e4a75c77a48ed7b742`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:28:22.662868 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-5fc2eba2-0228-4283-babc-e09f0e53b739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:22.731369 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1266/2558] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:28:22 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:28:22.742294 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83265e-ec75-4d0f-8df9-a00adea2eac8 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:22.744040 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ec83265e-ec75-4d0f-8df9-a00adea2eac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:22.802554 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83265e-ec75-4d0f-8df9-a00adea2eac8 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:22.806235 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0b663-34c0-49b3-a502-006a868839cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:22.806850 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0b663-34c0-49b3-a502-006a868839cf service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:22.807129 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0b663-34c0-49b3-a502-006a868839cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:22.807399 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0b663-34c0-49b3-a502-006a868839cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:22.817869 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee0b663-34c0-49b3-a502-006a868839cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:22.819324 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee0b663-34c0-49b3-a502-006a868839cf service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:22.823260 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:22.831240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1293/2559] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:28:22 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:28:22.969210 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc2eba2-0228-4283-babc-e09f0e53b739 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:23.031887 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc2eba2-0228-4283-babc-e09f0e53b739 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:23.036994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3b2e9-45c0-455d-812c-5d919eb7ff5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:23.037627 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3b2e9-45c0-455d-812c-5d919eb7ff5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:23.037912 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3b2e9-45c0-455d-812c-5d919eb7ff5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:23.038191 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3b2e9-45c0-455d-812c-5d919eb7ff5c admin admin] PATH_INFO: `/v3/users/78a7046d6de64d849abecd4c9f6ef5e2` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:23.039320 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba3b2e9-45c0-455d-812c-5d919eb7ff5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=78a7046d6de64d849abecd4c9f6ef5e2)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:23.041286 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba3b2e9-45c0-455d-812c-5d919eb7ff5c admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:23.087227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8ba3b2e9-45c0-455d-812c-5d919eb7ff5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78a7046d6de64d849abecd4c9f6ef5e2'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:23.089752 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:23.096553 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8ba3b2e9-45c0-455d-812c-5d919eb7ff5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78a7046d6de64d849abecd4c9f6ef5e2'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:23.097381 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8ba3b2e9-45c0-455d-812c-5d919eb7ff5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78a7046d6de64d849abecd4c9f6ef5e2'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:23.112031 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8ba3b2e9-45c0-455d-812c-5d919eb7ff5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78a7046d6de64d849abecd4c9f6ef5e2'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:23.119235 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8ba3b2e9-45c0-455d-812c-5d919eb7ff5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78a7046d6de64d849abecd4c9f6ef5e2'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:23.125518 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1267/2560] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:28:22 2022] DELETE /identity/v3/users/78a7046d6de64d849abecd4c9f6ef5e2 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:23.130108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca10bb-4546-498b-8b5f-b935da7e740e None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:23.130584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca10bb-4546-498b-8b5f-b935da7e740e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:23.130715 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca10bb-4546-498b-8b5f-b935da7e740e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:23.148927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-83ca10bb-4546-498b-8b5f-b935da7e740e None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:28:23.150098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-83ca10bb-4546-498b-8b5f-b935da7e740e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:23.207038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1294/2561] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:28:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:28:23.217473 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3b2e9-45c0-455d-812c-5d919eb7ff5c admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:23.218882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-8ba3b2e9-45c0-455d-812c-5d919eb7ff5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:23.275505 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3b2e9-45c0-455d-812c-5d919eb7ff5c admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:23.279920 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d44270-aaea-4c53-b6c3-08898364ea88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:23.280514 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d44270-aaea-4c53-b6c3-08898364ea88 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:23.280790 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d44270-aaea-4c53-b6c3-08898364ea88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:23.281058 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d44270-aaea-4c53-b6c3-08898364ea88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:23.289974 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d44270-aaea-4c53-b6c3-08898364ea88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:23.291433 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d44270-aaea-4c53-b6c3-08898364ea88 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:23.301627 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1268/2562] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:28:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:28:23.564388 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca10bb-4546-498b-8b5f-b935da7e740e None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:23.566015 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-83ca10bb-4546-498b-8b5f-b935da7e740e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:23.582873 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca10bb-4546-498b-8b5f-b935da7e740e None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:23.587819 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2d600-2e55-42d1-853b-916fb6f1e342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:23.588510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2d600-2e55-42d1-853b-916fb6f1e342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:23.588775 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2d600-2e55-42d1-853b-916fb6f1e342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:23.589178 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2d600-2e55-42d1-853b-916fb6f1e342 admin admin] PATH_INFO: `/v3/projects/e54af3f9df6c43638e4cdf20df1d0711` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:23.590940 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2d600-2e55-42d1-853b-916fb6f1e342 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e54af3f9df6c43638e4cdf20df1d0711)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:23.592975 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2d600-2e55-42d1-853b-916fb6f1e342 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:23.639167 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0bc2d600-2e55-42d1-853b-916fb6f1e342 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e54af3f9df6c43638e4cdf20df1d0711', 'request_id': 'req-0bc2d600-2e55-42d1-853b-916fb6f1e342'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:23.644530 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0bc2d600-2e55-42d1-853b-916fb6f1e342 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e54af3f9df6c43638e4cdf20df1d0711', 'request_id': 'req-0bc2d600-2e55-42d1-853b-916fb6f1e342'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:23.650645 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0bc2d600-2e55-42d1-853b-916fb6f1e342 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e54af3f9df6c43638e4cdf20df1d0711', 'request_id': 'req-0bc2d600-2e55-42d1-853b-916fb6f1e342'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:23.663722 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0bc2d600-2e55-42d1-853b-916fb6f1e342 admin admin] The token cache is being invalidate because project e54af3f9df6c43638e4cdf20df1d0711 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:28:23.664015 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0bc2d600-2e55-42d1-853b-916fb6f1e342 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:23.664672 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0bc2d600-2e55-42d1-853b-916fb6f1e342 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:23.666143 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1295/2563] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:28:23 2022] DELETE /identity/v3/projects/e54af3f9df6c43638e4cdf20df1d0711 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:23.670147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d44270-aaea-4c53-b6c3-08898364ea88 service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:23.679936 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d44270-aaea-4c53-b6c3-08898364ea88 service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:23.738619 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c38412-dd6a-49d2-983c-5102e9fd462a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:23.739241 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c38412-dd6a-49d2-983c-5102e9fd462a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:23.739557 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c38412-dd6a-49d2-983c-5102e9fd462a admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:23.739811 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c38412-dd6a-49d2-983c-5102e9fd462a admin admin] PATH_INFO: `/v3/users/c35423ad984d43e4a75c77a48ed7b742` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:23.740923 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c38412-dd6a-49d2-983c-5102e9fd462a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c35423ad984d43e4a75c77a48ed7b742)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:23.742828 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c38412-dd6a-49d2-983c-5102e9fd462a admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:23.794302 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-38c38412-dd6a-49d2-983c-5102e9fd462a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c35423ad984d43e4a75c77a48ed7b742'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:23.802814 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-38c38412-dd6a-49d2-983c-5102e9fd462a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c35423ad984d43e4a75c77a48ed7b742'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:23.803610 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-38c38412-dd6a-49d2-983c-5102e9fd462a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c35423ad984d43e4a75c77a48ed7b742'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:23.816482 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-38c38412-dd6a-49d2-983c-5102e9fd462a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c35423ad984d43e4a75c77a48ed7b742'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:23.817648 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:23.826458 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-38c38412-dd6a-49d2-983c-5102e9fd462a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c35423ad984d43e4a75c77a48ed7b742'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:23.832817 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1269/2564] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:28:23 2022] DELETE /identity/v3/users/c35423ad984d43e4a75c77a48ed7b742 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:24.099668 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2d600-2e55-42d1-853b-916fb6f1e342 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:24.101823 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-0bc2d600-2e55-42d1-853b-916fb6f1e342 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:24.121975 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2d600-2e55-42d1-853b-916fb6f1e342 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:24.178648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafd6da-58a0-4e55-b3ea-f68475076d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:24.179333 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafd6da-58a0-4e55-b3ea-f68475076d9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:24.179475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafd6da-58a0-4e55-b3ea-f68475076d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:24.179745 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafd6da-58a0-4e55-b3ea-f68475076d9a admin admin] PATH_INFO: `/v3/projects/169d4b33178640ecb336eb9615951b4c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:24.180926 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaafd6da-58a0-4e55-b3ea-f68475076d9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=169d4b33178640ecb336eb9615951b4c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:24.182815 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaafd6da-58a0-4e55-b3ea-f68475076d9a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:24.207399 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:24.229870 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-aaafd6da-58a0-4e55-b3ea-f68475076d9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '169d4b33178640ecb336eb9615951b4c', 'request_id': 'req-aaafd6da-58a0-4e55-b3ea-f68475076d9a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:24.235927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-aaafd6da-58a0-4e55-b3ea-f68475076d9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '169d4b33178640ecb336eb9615951b4c', 'request_id': 'req-aaafd6da-58a0-4e55-b3ea-f68475076d9a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:24.242530 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-aaafd6da-58a0-4e55-b3ea-f68475076d9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '169d4b33178640ecb336eb9615951b4c', 'request_id': 'req-aaafd6da-58a0-4e55-b3ea-f68475076d9a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:24.256905 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-aaafd6da-58a0-4e55-b3ea-f68475076d9a admin admin] The token cache is being invalidate because project 169d4b33178640ecb336eb9615951b4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:28:24.257193 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-aaafd6da-58a0-4e55-b3ea-f68475076d9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:24.257864 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-aaafd6da-58a0-4e55-b3ea-f68475076d9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:24.259411 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1296/2565] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:28:24 2022] DELETE /identity/v3/projects/169d4b33178640ecb336eb9615951b4c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:24.264378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f7d86-13b9-4bd7-9d22-c55429936e68 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:24.264854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f7d86-13b9-4bd7-9d22-c55429936e68 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:24.264967 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f7d86-13b9-4bd7-9d22-c55429936e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:24.284981 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-5c3f7d86-13b9-4bd7-9d22-c55429936e68 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:28:24.286185 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-5c3f7d86-13b9-4bd7-9d22-c55429936e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:24.346153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1270/2566] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:28:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:28:24.354342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcffb77-3476-49d7-bbff-98eff18fdd50 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:24.354624 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcffb77-3476-49d7-bbff-98eff18fdd50 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:24.354901 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcffb77-3476-49d7-bbff-98eff18fdd50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:24.375492 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-3dcffb77-3476-49d7-bbff-98eff18fdd50 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:28:24.376763 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3dcffb77-3476-49d7-bbff-98eff18fdd50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:24.384686 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1297/2567] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:28:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:28:24.389732 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f7d86-13b9-4bd7-9d22-c55429936e68 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:24.400405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f7d86-13b9-4bd7-9d22-c55429936e68 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:24.405531 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a91ca5f-c96c-4415-a882-f073aae0ae91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:24.406125 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91ca5f-c96c-4415-a882-f073aae0ae91 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:24.406403 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91ca5f-c96c-4415-a882-f073aae0ae91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:24.406674 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91ca5f-c96c-4415-a882-f073aae0ae91 admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:24.407368 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a91ca5f-c96c-4415-a882-f073aae0ae91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:28:24.407660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a91ca5f-c96c-4415-a882-f073aae0ae91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:24.409265 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a91ca5f-c96c-4415-a882-f073aae0ae91 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:24.421521 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1271/2568] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:28:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:28:24.425918 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcffb77-3476-49d7-bbff-98eff18fdd50 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:24.435769 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcffb77-3476-49d7-bbff-98eff18fdd50 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:24.440407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac84706-b047-4709-b1d2-dffd64e7dec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:24.440993 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac84706-b047-4709-b1d2-dffd64e7dec0 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:24.441262 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac84706-b047-4709-b1d2-dffd64e7dec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:24.441527 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac84706-b047-4709-b1d2-dffd64e7dec0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:24.442155 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac84706-b047-4709-b1d2-dffd64e7dec0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:24.444066 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac84706-b047-4709-b1d2-dffd64e7dec0 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:24.470359 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1298/2569] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:28:24 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:28:24.474577 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a91ca5f-c96c-4415-a882-f073aae0ae91 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:24.485189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a91ca5f-c96c-4415-a882-f073aae0ae91 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:24.542409 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4486c-c190-4214-90bf-084ee1474f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:24.543592 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4486c-c190-4214-90bf-084ee1474f56 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:24.543855 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4486c-c190-4214-90bf-084ee1474f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:24.544142 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4486c-c190-4214-90bf-084ee1474f56 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:24.544762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4486c-c190-4214-90bf-084ee1474f56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:24.546661 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4486c-c190-4214-90bf-084ee1474f56 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:24.569730 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1272/2570] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:28:24 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:28:24.573821 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac84706-b047-4709-b1d2-dffd64e7dec0 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:24.584086 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac84706-b047-4709-b1d2-dffd64e7dec0 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:24.588649 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b50c8d4-a338-45a3-b682-a5e270cf556b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:24.589232 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50c8d4-a338-45a3-b682-a5e270cf556b admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:24.589505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50c8d4-a338-45a3-b682-a5e270cf556b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:24.590252 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50c8d4-a338-45a3-b682-a5e270cf556b admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:24.590894 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b50c8d4-a338-45a3-b682-a5e270cf556b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:24.592492 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b50c8d4-a338-45a3-b682-a5e270cf556b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:24.602168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1299/2571] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:28:24.606705 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4486c-c190-4214-90bf-084ee1474f56 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:24.616560 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4486c-c190-4214-90bf-084ee1474f56 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:24.621163 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9516a56-6de9-4fe7-bbeb-02a97da6419e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:24.621747 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9516a56-6de9-4fe7-bbeb-02a97da6419e admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:24.622024 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9516a56-6de9-4fe7-bbeb-02a97da6419e admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:24.622293 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9516a56-6de9-4fe7-bbeb-02a97da6419e admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:24.622929 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9516a56-6de9-4fe7-bbeb-02a97da6419e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:24.624532 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9516a56-6de9-4fe7-bbeb-02a97da6419e admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:24.635031 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1273/2572] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:28:24.639285 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b50c8d4-a338-45a3-b682-a5e270cf556b admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:24.649606 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b50c8d4-a338-45a3-b682-a5e270cf556b admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:24.654976 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e4d1b-e851-45b7-8f78-c06308a63e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:24.655664 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e4d1b-e851-45b7-8f78-c06308a63e2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:24.655949 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e4d1b-e851-45b7-8f78-c06308a63e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:24.656225 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e4d1b-e851-45b7-8f78-c06308a63e2a admin admin] PATH_INFO: `/v3/projects/68cd48e67cf343d9b91a69dcbded822d/users/253f01e533614147a5fbc5263c8c79d6/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:24.671622 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e4d1b-e851-45b7-8f78-c06308a63e2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=68cd48e67cf343d9b91a69dcbded822d, user_id=253f01e533614147a5fbc5263c8c79d6, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:24.674281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e4d1b-e851-45b7-8f78-c06308a63e2a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:24.688520 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1300/2573] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:28:24 2022] PUT /identity/v3/projects/68cd48e67cf343d9b91a69dcbded822d/users/253f01e533614147a5fbc5263c8c79d6/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:24.698612 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081bfbe-7ebf-4dfc-8b2c-bc5d2f31a851 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:24.698933 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081bfbe-7ebf-4dfc-8b2c-bc5d2f31a851 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:24.699211 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081bfbe-7ebf-4dfc-8b2c-bc5d2f31a851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:24.727023 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-0081bfbe-7ebf-4dfc-8b2c-bc5d2f31a851 None None] MFA Rules not processed for user `253f01e533614147a5fbc5263c8c79d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:28:24.728272 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-0081bfbe-7ebf-4dfc-8b2c-bc5d2f31a851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:24.773040 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:24.795960 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1274/2574] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:28:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:28:24.800539 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e4d1b-e851-45b7-8f78-c06308a63e2a admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:24.860638 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e4d1b-e851-45b7-8f78-c06308a63e2a admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:24.865394 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb0a71-c5c2-4993-93fc-99f16c39da6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:24.865997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb0a71-c5c2-4993-93fc-99f16c39da6a admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:24.866282 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb0a71-c5c2-4993-93fc-99f16c39da6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:24.866553 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb0a71-c5c2-4993-93fc-99f16c39da6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:24.867185 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb0a71-c5c2-4993-93fc-99f16c39da6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:24.869129 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb0a71-c5c2-4993-93fc-99f16c39da6a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:24.896047 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1301/2575] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:28:24 2022] POST /identity/v3/projects => generated 373 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:28:24.900176 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081bfbe-7ebf-4dfc-8b2c-bc5d2f31a851 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:24.911804 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081bfbe-7ebf-4dfc-8b2c-bc5d2f31a851 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:24.968415 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dbfaed-817e-43a8-95b1-7c1ccbc0a656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:24.968955 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbfaed-817e-43a8-95b1-7c1ccbc0a656 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:24.969238 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbfaed-817e-43a8-95b1-7c1ccbc0a656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:24.969514 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbfaed-817e-43a8-95b1-7c1ccbc0a656 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:24.970152 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dbfaed-817e-43a8-95b1-7c1ccbc0a656 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:24.972170 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dbfaed-817e-43a8-95b1-7c1ccbc0a656 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:24.993058 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1275/2576] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:28:24 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:28:24.997057 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb0a71-c5c2-4993-93fc-99f16c39da6a admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:25.007412 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb0a71-c5c2-4993-93fc-99f16c39da6a admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:25.012593 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee9ddc-d24f-4d7d-8dad-fc89f020aabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:25.013182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee9ddc-d24f-4d7d-8dad-fc89f020aabb admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:25.013462 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee9ddc-d24f-4d7d-8dad-fc89f020aabb admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:25.013732 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee9ddc-d24f-4d7d-8dad-fc89f020aabb admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:25.014366 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee9ddc-d24f-4d7d-8dad-fc89f020aabb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:25.016215 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee9ddc-d24f-4d7d-8dad-fc89f020aabb admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:25.027069 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1302/2577] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:28:25.031919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dbfaed-817e-43a8-95b1-7c1ccbc0a656 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:25.041783 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dbfaed-817e-43a8-95b1-7c1ccbc0a656 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:25.046357 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7f2d1-ffe5-47e3-acc8-743b5a316d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:25.046996 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7f2d1-ffe5-47e3-acc8-743b5a316d51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:25.047274 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7f2d1-ffe5-47e3-acc8-743b5a316d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:25.047571 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7f2d1-ffe5-47e3-acc8-743b5a316d51 admin admin] PATH_INFO: `/v3/projects/dba57b1db19143d88d86c7e88c5c9f0b/users/0c9fd200f55c4116a25b909d07434685/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:25.062371 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a7f2d1-ffe5-47e3-acc8-743b5a316d51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dba57b1db19143d88d86c7e88c5c9f0b, user_id=0c9fd200f55c4116a25b909d07434685, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:25.065047 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a7f2d1-ffe5-47e3-acc8-743b5a316d51 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:25.083268 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1276/2578] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:28:25 2022] PUT /identity/v3/projects/dba57b1db19143d88d86c7e88c5c9f0b/users/0c9fd200f55c4116a25b909d07434685/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:28:25.087174 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee9ddc-d24f-4d7d-8dad-fc89f020aabb admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:25.098432 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee9ddc-d24f-4d7d-8dad-fc89f020aabb admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:25.116167 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:25.156150 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb19e69-8258-4ee1-97eb-86c589bf9ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:25.156756 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb19e69-8258-4ee1-97eb-86c589bf9ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:25.157035 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb19e69-8258-4ee1-97eb-86c589bf9ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:25.157308 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb19e69-8258-4ee1-97eb-86c589bf9ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:25.157944 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb19e69-8258-4ee1-97eb-86c589bf9ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:25.159596 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb19e69-8258-4ee1-97eb-86c589bf9ce0 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:25.169643 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1303/2579] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:28:25.174071 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7f2d1-ffe5-47e3-acc8-743b5a316d51 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:25.185003 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7f2d1-ffe5-47e3-acc8-743b5a316d51 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:25.189691 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64bcf8-0f30-4d82-9af7-268a77b9610d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:25.190627 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64bcf8-0f30-4d82-9af7-268a77b9610d admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:25.191026 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64bcf8-0f30-4d82-9af7-268a77b9610d admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:25.191431 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64bcf8-0f30-4d82-9af7-268a77b9610d admin admin] PATH_INFO: `/v3/projects/dba57b1db19143d88d86c7e88c5c9f0b/users/0c9fd200f55c4116a25b909d07434685/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:25.193599 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e64bcf8-0f30-4d82-9af7-268a77b9610d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dba57b1db19143d88d86c7e88c5c9f0b, user_id=0c9fd200f55c4116a25b909d07434685, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:25.196267 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e64bcf8-0f30-4d82-9af7-268a77b9610d admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:25.208388 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1277/2580] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:28:25 2022] PUT /identity/v3/projects/dba57b1db19143d88d86c7e88c5c9f0b/users/0c9fd200f55c4116a25b909d07434685/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:25.220301 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80649a-ab9e-449f-8521-283025c9aa26 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:25.220787 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80649a-ab9e-449f-8521-283025c9aa26 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:25.220787 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80649a-ab9e-449f-8521-283025c9aa26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:25.249902 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-1b80649a-ab9e-449f-8521-283025c9aa26 None None] MFA Rules not processed for user `0c9fd200f55c4116a25b909d07434685`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:28:25.251222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1b80649a-ab9e-449f-8521-283025c9aa26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:25.319200 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1304/2581] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:28:25 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:28:25.331765 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64bcf8-0f30-4d82-9af7-268a77b9610d admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:25.333366 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-1e64bcf8-0f30-4d82-9af7-268a77b9610d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:25.391493 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64bcf8-0f30-4d82-9af7-268a77b9610d admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:25.395150 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06629d0d-2786-4903-8060-c2ec053a4f23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:25.395781 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06629d0d-2786-4903-8060-c2ec053a4f23 service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:25.396062 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06629d0d-2786-4903-8060-c2ec053a4f23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:25.396334 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06629d0d-2786-4903-8060-c2ec053a4f23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:25.444883 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:25.455404 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06629d0d-2786-4903-8060-c2ec053a4f23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:25.456973 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06629d0d-2786-4903-8060-c2ec053a4f23 service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:25.466626 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1278/2582] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:28:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:28:28.003939 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b80649a-ab9e-449f-8521-283025c9aa26 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:28.064537 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b80649a-ab9e-449f-8521-283025c9aa26 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:28.069001 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1848975-8f09-4dbe-967b-c57c85b7caed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:28.069622 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1848975-8f09-4dbe-967b-c57c85b7caed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:28.069933 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1848975-8f09-4dbe-967b-c57c85b7caed admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:28.070171 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1848975-8f09-4dbe-967b-c57c85b7caed admin admin] PATH_INFO: `/v3/users/253f01e533614147a5fbc5263c8c79d6` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:28.071272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1848975-8f09-4dbe-967b-c57c85b7caed admin admin] RBAC: Authorizing `identity:delete_user(user_id=253f01e533614147a5fbc5263c8c79d6)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:28.073219 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1848975-8f09-4dbe-967b-c57c85b7caed admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:28.116424 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:28.118134 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a1848975-8f09-4dbe-967b-c57c85b7caed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '253f01e533614147a5fbc5263c8c79d6'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:28.127047 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a1848975-8f09-4dbe-967b-c57c85b7caed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '253f01e533614147a5fbc5263c8c79d6'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:28.127771 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a1848975-8f09-4dbe-967b-c57c85b7caed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '253f01e533614147a5fbc5263c8c79d6'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:28.128370 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a1848975-8f09-4dbe-967b-c57c85b7caed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '253f01e533614147a5fbc5263c8c79d6'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:28.140363 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a1848975-8f09-4dbe-967b-c57c85b7caed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '253f01e533614147a5fbc5263c8c79d6'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:28.152635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1305/2583] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:28:28 2022] DELETE /identity/v3/users/253f01e533614147a5fbc5263c8c79d6 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:28.157496 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66dbc3-035d-4c84-b708-5fcc5e2d679e None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:28.157990 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66dbc3-035d-4c84-b708-5fcc5e2d679e None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:28.158100 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66dbc3-035d-4c84-b708-5fcc5e2d679e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:28.178238 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-ca66dbc3-035d-4c84-b708-5fcc5e2d679e None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:28:28.179434 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-ca66dbc3-035d-4c84-b708-5fcc5e2d679e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:28.239251 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1279/2584] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:28:28.248983 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1848975-8f09-4dbe-967b-c57c85b7caed admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:28.250419 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a1848975-8f09-4dbe-967b-c57c85b7caed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:28.309642 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1848975-8f09-4dbe-967b-c57c85b7caed admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:28.313384 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6331225-88f4-4b7c-8e77-f8b6e5a26b41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:28.313983 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6331225-88f4-4b7c-8e77-f8b6e5a26b41 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:28.314265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6331225-88f4-4b7c-8e77-f8b6e5a26b41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:28.314539 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6331225-88f4-4b7c-8e77-f8b6e5a26b41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:28.323753 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6331225-88f4-4b7c-8e77-f8b6e5a26b41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:28.325295 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6331225-88f4-4b7c-8e77-f8b6e5a26b41 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:28.334714 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1306/2585] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:28:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:28:28.611696 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66dbc3-035d-4c84-b708-5fcc5e2d679e None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:28.613162 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-ca66dbc3-035d-4c84-b708-5fcc5e2d679e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:28.627975 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66dbc3-035d-4c84-b708-5fcc5e2d679e None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:28.632769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500c3ed-3f58-49be-9f96-44e60ce97416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:28.633402 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500c3ed-3f58-49be-9f96-44e60ce97416 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:28.633688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500c3ed-3f58-49be-9f96-44e60ce97416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:28.633961 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500c3ed-3f58-49be-9f96-44e60ce97416 admin admin] PATH_INFO: `/v3/projects/68cd48e67cf343d9b91a69dcbded822d` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:28.635296 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500c3ed-3f58-49be-9f96-44e60ce97416 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68cd48e67cf343d9b91a69dcbded822d)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:28.637448 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500c3ed-3f58-49be-9f96-44e60ce97416 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:28.686856 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b500c3ed-3f58-49be-9f96-44e60ce97416 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68cd48e67cf343d9b91a69dcbded822d', 'request_id': 'req-b500c3ed-3f58-49be-9f96-44e60ce97416'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:28.695283 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b500c3ed-3f58-49be-9f96-44e60ce97416 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68cd48e67cf343d9b91a69dcbded822d', 'request_id': 'req-b500c3ed-3f58-49be-9f96-44e60ce97416'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:28.702549 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b500c3ed-3f58-49be-9f96-44e60ce97416 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68cd48e67cf343d9b91a69dcbded822d', 'request_id': 'req-b500c3ed-3f58-49be-9f96-44e60ce97416'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:28.713982 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b500c3ed-3f58-49be-9f96-44e60ce97416 admin admin] The token cache is being invalidate because project 68cd48e67cf343d9b91a69dcbded822d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:28:28.714260 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b500c3ed-3f58-49be-9f96-44e60ce97416 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:28.714909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b500c3ed-3f58-49be-9f96-44e60ce97416 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:28.717049 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1280/2586] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:28:28 2022] DELETE /identity/v3/projects/68cd48e67cf343d9b91a69dcbded822d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:28.721280 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6331225-88f4-4b7c-8e77-f8b6e5a26b41 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:28.731007 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6331225-88f4-4b7c-8e77-f8b6e5a26b41 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:28.798575 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ff2b4-7882-4ac1-9578-f65152bc9a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:28.799244 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ff2b4-7882-4ac1-9578-f65152bc9a5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:28.799546 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ff2b4-7882-4ac1-9578-f65152bc9a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:28.799829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ff2b4-7882-4ac1-9578-f65152bc9a5b admin admin] PATH_INFO: `/v3/users/0c9fd200f55c4116a25b909d07434685` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:28.800979 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ff2b4-7882-4ac1-9578-f65152bc9a5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c9fd200f55c4116a25b909d07434685)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:28.802897 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ff2b4-7882-4ac1-9578-f65152bc9a5b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:28.849298 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2d7ff2b4-7882-4ac1-9578-f65152bc9a5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c9fd200f55c4116a25b909d07434685'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:28.855991 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2d7ff2b4-7882-4ac1-9578-f65152bc9a5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c9fd200f55c4116a25b909d07434685'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:28.856645 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2d7ff2b4-7882-4ac1-9578-f65152bc9a5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c9fd200f55c4116a25b909d07434685'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:28.857256 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2d7ff2b4-7882-4ac1-9578-f65152bc9a5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c9fd200f55c4116a25b909d07434685'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:28.864898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:28.870295 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2d7ff2b4-7882-4ac1-9578-f65152bc9a5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c9fd200f55c4116a25b909d07434685'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:28.883301 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1307/2587] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:28:28 2022] DELETE /identity/v3/users/0c9fd200f55c4116a25b909d07434685 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:28:29.149395 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500c3ed-3f58-49be-9f96-44e60ce97416 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:29.151421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-b500c3ed-3f58-49be-9f96-44e60ce97416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:29.165360 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500c3ed-3f58-49be-9f96-44e60ce97416 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:29.224189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d85f1-f486-4d71-a2c2-17ae74ac8ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:29.224846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d85f1-f486-4d71-a2c2-17ae74ac8ca6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:29.225137 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d85f1-f486-4d71-a2c2-17ae74ac8ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:29.225401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d85f1-f486-4d71-a2c2-17ae74ac8ca6 admin admin] PATH_INFO: `/v3/projects/dba57b1db19143d88d86c7e88c5c9f0b` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:29.226604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d85f1-f486-4d71-a2c2-17ae74ac8ca6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dba57b1db19143d88d86c7e88c5c9f0b)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:29.228976 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d85f1-f486-4d71-a2c2-17ae74ac8ca6 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:29.250252 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:29.277304 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-478d85f1-f486-4d71-a2c2-17ae74ac8ca6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dba57b1db19143d88d86c7e88c5c9f0b', 'request_id': 'req-478d85f1-f486-4d71-a2c2-17ae74ac8ca6'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:29.284761 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-478d85f1-f486-4d71-a2c2-17ae74ac8ca6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dba57b1db19143d88d86c7e88c5c9f0b', 'request_id': 'req-478d85f1-f486-4d71-a2c2-17ae74ac8ca6'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:29.291816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-478d85f1-f486-4d71-a2c2-17ae74ac8ca6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dba57b1db19143d88d86c7e88c5c9f0b', 'request_id': 'req-478d85f1-f486-4d71-a2c2-17ae74ac8ca6'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:29.304334 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-478d85f1-f486-4d71-a2c2-17ae74ac8ca6 admin admin] The token cache is being invalidate because project dba57b1db19143d88d86c7e88c5c9f0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:28:29.304768 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-478d85f1-f486-4d71-a2c2-17ae74ac8ca6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:29.305588 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-478d85f1-f486-4d71-a2c2-17ae74ac8ca6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:28:29.307129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1281/2588] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:28:29 2022] DELETE /identity/v3/projects/dba57b1db19143d88d86c7e88c5c9f0b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:29.313064 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e07bba-850f-4cc5-b608-aa8878e0327d None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:29.313542 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e07bba-850f-4cc5-b608-aa8878e0327d None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:29.313640 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e07bba-850f-4cc5-b608-aa8878e0327d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:29.334398 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-55e07bba-850f-4cc5-b608-aa8878e0327d None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:28:29.335719 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-55e07bba-850f-4cc5-b608-aa8878e0327d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:29.404066 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1308/2589] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:28:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:28:29.411787 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36284a-0c13-419e-a60e-a54f2f7b0dd0 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:29.412068 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36284a-0c13-419e-a60e-a54f2f7b0dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:29.412339 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36284a-0c13-419e-a60e-a54f2f7b0dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:29.434591 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-3f36284a-0c13-419e-a60e-a54f2f7b0dd0 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:28:29.436537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-3f36284a-0c13-419e-a60e-a54f2f7b0dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:29.444803 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1282/2590] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:28:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:28:29.448880 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e07bba-850f-4cc5-b608-aa8878e0327d None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:29.459010 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e07bba-850f-4cc5-b608-aa8878e0327d None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:29.463622 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e0cbb-dd5c-465f-b14d-8988af22faac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:29.464219 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e0cbb-dd5c-465f-b14d-8988af22faac admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:29.464502 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e0cbb-dd5c-465f-b14d-8988af22faac admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:29.464808 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e0cbb-dd5c-465f-b14d-8988af22faac admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:29.465452 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e0cbb-dd5c-465f-b14d-8988af22faac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:28:29.465757 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e0cbb-dd5c-465f-b14d-8988af22faac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:29.467373 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e0cbb-dd5c-465f-b14d-8988af22faac admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:29.479738 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1309/2591] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:28:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:28:29.485319 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f36284a-0c13-419e-a60e-a54f2f7b0dd0 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:29.496105 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f36284a-0c13-419e-a60e-a54f2f7b0dd0 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:29.502226 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c9184-904b-49a1-822b-06c5f9a901cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:29.502979 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c9184-904b-49a1-822b-06c5f9a901cb admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:29.503258 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c9184-904b-49a1-822b-06c5f9a901cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:29.503564 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c9184-904b-49a1-822b-06c5f9a901cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:29.504396 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c9184-904b-49a1-822b-06c5f9a901cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:29.506312 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c9184-904b-49a1-822b-06c5f9a901cb admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:29.538274 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1283/2592] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:28:29 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:28:29.542579 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e0cbb-dd5c-465f-b14d-8988af22faac admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:29.554722 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e0cbb-dd5c-465f-b14d-8988af22faac admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:29.616251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042dd1b-4c68-4611-9328-561706abacec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:29.616868 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042dd1b-4c68-4611-9328-561706abacec admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:29.617150 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042dd1b-4c68-4611-9328-561706abacec admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:29.617421 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042dd1b-4c68-4611-9328-561706abacec admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:29.618066 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7042dd1b-4c68-4611-9328-561706abacec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:29.620012 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7042dd1b-4c68-4611-9328-561706abacec admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:29.641926 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1310/2593] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:28:29 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:28:29.645973 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c9184-904b-49a1-822b-06c5f9a901cb admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:29.656581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c9184-904b-49a1-822b-06c5f9a901cb admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:29.661451 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9186707-6bf8-4a8d-9731-f8497ba5f009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:29.662097 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9186707-6bf8-4a8d-9731-f8497ba5f009 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:29.662376 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9186707-6bf8-4a8d-9731-f8497ba5f009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:29.662648 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9186707-6bf8-4a8d-9731-f8497ba5f009 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:29.663320 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9186707-6bf8-4a8d-9731-f8497ba5f009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:29.665062 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9186707-6bf8-4a8d-9731-f8497ba5f009 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:29.676125 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1284/2594] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:28:29.680258 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042dd1b-4c68-4611-9328-561706abacec admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:29.691023 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042dd1b-4c68-4611-9328-561706abacec admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:29.695768 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3accbfcb-9365-48c9-bea8-791a360521f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:29.696370 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3accbfcb-9365-48c9-bea8-791a360521f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:29.696652 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3accbfcb-9365-48c9-bea8-791a360521f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:29.697026 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3accbfcb-9365-48c9-bea8-791a360521f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:29.697557 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3accbfcb-9365-48c9-bea8-791a360521f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:29.699133 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3accbfcb-9365-48c9-bea8-791a360521f5 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:29.710033 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1311/2595] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:28:29.714123 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9186707-6bf8-4a8d-9731-f8497ba5f009 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:29.724744 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9186707-6bf8-4a8d-9731-f8497ba5f009 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:29.729732 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4927da-6069-426e-b310-edf97b1f8f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:29.730522 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4927da-6069-426e-b310-edf97b1f8f96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:29.730657 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4927da-6069-426e-b310-edf97b1f8f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:29.730911 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4927da-6069-426e-b310-edf97b1f8f96 admin admin] PATH_INFO: `/v3/projects/1645a9dd82b14c25892cec9be947f310/users/51ad1f400be2465dbfb4a14698247496/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:29.746502 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4927da-6069-426e-b310-edf97b1f8f96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1645a9dd82b14c25892cec9be947f310, user_id=51ad1f400be2465dbfb4a14698247496, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:29.749288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4927da-6069-426e-b310-edf97b1f8f96 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:29.763150 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1285/2596] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:28:29 2022] PUT /identity/v3/projects/1645a9dd82b14c25892cec9be947f310/users/51ad1f400be2465dbfb4a14698247496/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:28:29.767984 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf76247-926a-4381-84df-5e3074dc0292 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:29.768270 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf76247-926a-4381-84df-5e3074dc0292 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:29.768546 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf76247-926a-4381-84df-5e3074dc0292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:29.788184 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-ccf76247-926a-4381-84df-5e3074dc0292 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:28:29.789390 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ccf76247-926a-4381-84df-5e3074dc0292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:29.835766 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:29.851798 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1312/2597] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:28:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:28:29.862102 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4927da-6069-426e-b310-edf97b1f8f96 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:29.863822 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-eb4927da-6069-426e-b310-edf97b1f8f96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:29.920753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4927da-6069-426e-b310-edf97b1f8f96 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:29.924675 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785eee7b-d8f0-4455-9932-378d6383679f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:29.925281 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785eee7b-d8f0-4455-9932-378d6383679f service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:29.925559 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785eee7b-d8f0-4455-9932-378d6383679f service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:29.925857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785eee7b-d8f0-4455-9932-378d6383679f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:29.934754 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785eee7b-d8f0-4455-9932-378d6383679f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:29.936268 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785eee7b-d8f0-4455-9932-378d6383679f service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:29.945664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1286/2598] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:28:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:28:30.564950 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bc5fd-f930-48b9-a07c-9f1fcec1ba95 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:30.565469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bc5fd-f930-48b9-a07c-9f1fcec1ba95 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:30.565554 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bc5fd-f930-48b9-a07c-9f1fcec1ba95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:30.594107 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-944bc5fd-f930-48b9-a07c-9f1fcec1ba95 None None] MFA Rules not processed for user `51ad1f400be2465dbfb4a14698247496`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:28:30.595246 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-944bc5fd-f930-48b9-a07c-9f1fcec1ba95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:30.664008 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1313/2599] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:28:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:28:30.674855 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785eee7b-d8f0-4455-9932-378d6383679f service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:30.676342 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-785eee7b-d8f0-4455-9932-378d6383679f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:30.726067 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:30.741441 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785eee7b-d8f0-4455-9932-378d6383679f service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:30.748595 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4454d0ab-7fd6-4a19-a409-8c98e31de5b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:30.749190 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4454d0ab-7fd6-4a19-a409-8c98e31de5b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:30.749471 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4454d0ab-7fd6-4a19-a409-8c98e31de5b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:30.749743 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4454d0ab-7fd6-4a19-a409-8c98e31de5b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:30.758615 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4454d0ab-7fd6-4a19-a409-8c98e31de5b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:30.760098 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4454d0ab-7fd6-4a19-a409-8c98e31de5b4 service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:30.769874 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1287/2600] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:28:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:28:32.137811 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce50400-1907-4846-9f24-be81c6769baf None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:32.138968 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce50400-1907-4846-9f24-be81c6769baf None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:32.138968 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce50400-1907-4846-9f24-be81c6769baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:32.157724 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-9ce50400-1907-4846-9f24-be81c6769baf None None] MFA Rules not processed for user `51ad1f400be2465dbfb4a14698247496`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:28:32.158908 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9ce50400-1907-4846-9f24-be81c6769baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:32.165996 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1314/2601] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:28:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:28:40.765727 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=6 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:42.152581 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=39 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:28:43.904356 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4454d0ab-7fd6-4a19-a409-8c98e31de5b4 service cinder] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:43.906324 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-4454d0ab-7fd6-4a19-a409-8c98e31de5b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:43.918118 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4454d0ab-7fd6-4a19-a409-8c98e31de5b4 service cinder] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:43.922418 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b62b2-1835-4d5a-b549-67b69c46aa47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:43.922993 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b62b2-1835-4d5a-b549-67b69c46aa47 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:43.923268 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b62b2-1835-4d5a-b549-67b69c46aa47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:43.923670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b62b2-1835-4d5a-b549-67b69c46aa47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:43.925207 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-0e9b62b2-1835-4d5a-b549-67b69c46aa47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:43.995204 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b62b2-1835-4d5a-b549-67b69c46aa47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:43.996665 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b62b2-1835-4d5a-b549-67b69c46aa47 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:44.007991 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1288/2602] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:28:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:28:48.775611 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce50400-1907-4846-9f24-be81c6769baf None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:48.786936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce50400-1907-4846-9f24-be81c6769baf None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:48.790768 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b4ee0-60cd-4a2a-bc22-f4f42ea06d30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:48.791381 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b4ee0-60cd-4a2a-bc22-f4f42ea06d30 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:48.791698 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b4ee0-60cd-4a2a-bc22-f4f42ea06d30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:48.792173 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b4ee0-60cd-4a2a-bc22-f4f42ea06d30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:48.793708 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f93b4ee0-60cd-4a2a-bc22-f4f42ea06d30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:48.808614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b4ee0-60cd-4a2a-bc22-f4f42ea06d30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:48.810196 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b4ee0-60cd-4a2a-bc22-f4f42ea06d30 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:48.827940 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1315/2603] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:28:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:28:49.244979 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca6fdf-3141-43d3-b18b-c935146255f9 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:49.245547 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca6fdf-3141-43d3-b18b-c935146255f9 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:49.245547 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca6fdf-3141-43d3-b18b-c935146255f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:49.267870 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-39ca6fdf-3141-43d3-b18b-c935146255f9 None None] MFA Rules not processed for user `8c5ae98336c441118d01f46a50f732d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:28:49.269078 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-39ca6fdf-3141-43d3-b18b-c935146255f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:49.331049 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1289/2604] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Sep 5 12:28:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:28:49.384265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b4ee0-60cd-4a2a-bc22-f4f42ea06d30 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:49.385921 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f93b4ee0-60cd-4a2a-bc22-f4f42ea06d30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:49.458626 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b4ee0-60cd-4a2a-bc22-f4f42ea06d30 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:49.464614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e64b5-cbd6-4410-adca-5af7b74a0029 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '857eb7657e7d45938baca17f91a16901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:49.464750 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e64b5-cbd6-4410-adca-5af7b74a0029 service swift] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:49.464911 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e64b5-cbd6-4410-adca-5af7b74a0029 service swift] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:49.465582 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e64b5-cbd6-4410-adca-5af7b74a0029 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:49.474589 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4e64b5-cbd6-4410-adca-5af7b74a0029 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:49.476211 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4e64b5-cbd6-4410-adca-5af7b74a0029 service swift] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:49.484958 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1316/2605] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Sep 5 12:28:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 12:28:50.037005 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbc8b9-bae6-470a-b7e3-2b79681047b4 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:50.037502 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbc8b9-bae6-470a-b7e3-2b79681047b4 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:50.039296 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbc8b9-bae6-470a-b7e3-2b79681047b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:50.248091 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-a3dbc8b9-bae6-470a-b7e3-2b79681047b4 None None] MFA Rules not processed for user `51ad1f400be2465dbfb4a14698247496`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:28:50.249285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-a3dbc8b9-bae6-470a-b7e3-2b79681047b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:50.257940 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1290/2606] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:28:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:28:52.550764 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e64b5-cbd6-4410-adca-5af7b74a0029 service swift] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:28:52.566725 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e64b5-cbd6-4410-adca-5af7b74a0029 service swift] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:28:52.574455 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba59be-6930-49f1-9932-92f18d58d380 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:28:52.575365 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba59be-6930-49f1-9932-92f18d58d380 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:28:52.575881 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba59be-6930-49f1-9932-92f18d58d380 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:28:52.576156 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba59be-6930-49f1-9932-92f18d58d380 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:28:52.578929 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1bba59be-6930-49f1-9932-92f18d58d380 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:28:52.713040 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba59be-6930-49f1-9932-92f18d58d380 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:28:52.720455 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba59be-6930-49f1-9932-92f18d58d380 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:28:52.744095 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1317/2607] 10.222.0.16 () {60 vars in 1396 bytes} [Mon Sep 5 12:28:52 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3266 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:00.073321 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=62 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:02.738289 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=79 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:10.488737 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9670b7-e4ff-418d-b79a-0bc21b98ee96 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:10.489314 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9670b7-e4ff-418d-b79a-0bc21b98ee96 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:10.489426 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9670b7-e4ff-418d-b79a-0bc21b98ee96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:10.516102 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-2d9670b7-e4ff-418d-b79a-0bc21b98ee96 None None] MFA Rules not processed for user `ebd8a00f21344c7990ffe1a7ff2986ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:10.517326 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-2d9670b7-e4ff-418d-b79a-0bc21b98ee96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:10.583688 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1291/2608] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:29:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:29:10.928910 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ba26c-cacb-43f8-a8e3-3e6763b762cc None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:10.929853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ba26c-cacb-43f8-a8e3-3e6763b762cc None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:10.930442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ba26c-cacb-43f8-a8e3-3e6763b762cc None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:10.932356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1318/2609] 10.222.0.16 () {54 vars in 921 bytes} [Mon Sep 5 12:29:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:29:10.948109 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89566481-2995-4c9d-aaa9-7d8d2570ea01 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:10.948783 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89566481-2995-4c9d-aaa9-7d8d2570ea01 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:10.949159 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89566481-2995-4c9d-aaa9-7d8d2570ea01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:11.025162 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-89566481-2995-4c9d-aaa9-7d8d2570ea01 None None] MFA Rules not processed for user `7cda290418504f0db85d665479a05353`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:11.027352 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-89566481-2995-4c9d-aaa9-7d8d2570ea01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:11.042348 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1292/2610] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Sep 5 12:29:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:11.065695 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ba26c-cacb-43f8-a8e3-3e6763b762cc None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:11.072535 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-fb6ba26c-cacb-43f8-a8e3-3e6763b762cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:11.147408 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ba26c-cacb-43f8-a8e3-3e6763b762cc None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:11.151495 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6a17f-1e7e-4e51-b9de-9694d5a693a8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed75c0be809d4a0eb0e2ccc59805b1ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:11.152488 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6a17f-1e7e-4e51-b9de-9694d5a693a8 service glance] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:11.152998 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6a17f-1e7e-4e51-b9de-9694d5a693a8 service glance] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:11.153277 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6a17f-1e7e-4e51-b9de-9694d5a693a8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:11.165259 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f6a17f-1e7e-4e51-b9de-9694d5a693a8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:11.166705 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f6a17f-1e7e-4e51-b9de-9694d5a693a8 service glance] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:11.191784 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1319/2611] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:16.093038 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89566481-2995-4c9d-aaa9-7d8d2570ea01 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:16.103395 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89566481-2995-4c9d-aaa9-7d8d2570ea01 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:16.106874 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75cf91-a394-401d-b0aa-6676e32f37da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:16.108052 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75cf91-a394-401d-b0aa-6676e32f37da service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:16.108336 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75cf91-a394-401d-b0aa-6676e32f37da service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:16.108604 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75cf91-a394-401d-b0aa-6676e32f37da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:16.110297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-6f75cf91-a394-401d-b0aa-6676e32f37da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:16.119764 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f75cf91-a394-401d-b0aa-6676e32f37da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:16.121211 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f75cf91-a394-401d-b0aa-6676e32f37da service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:16.129943 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1293/2612] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:17.193913 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf91adc-eae4-4287-9ddc-1811ad79c7e4 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:17.194428 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf91adc-eae4-4287-9ddc-1811ad79c7e4 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:17.194594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf91adc-eae4-4287-9ddc-1811ad79c7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:17.250175 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-edf91adc-eae4-4287-9ddc-1811ad79c7e4 None None] MFA Rules not processed for user `8c5ae98336c441118d01f46a50f732d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:17.251405 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-edf91adc-eae4-4287-9ddc-1811ad79c7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:17.270028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1320/2613] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Sep 5 12:29:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:17.282710 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75cf91-a394-401d-b0aa-6676e32f37da service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:17.306294 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75cf91-a394-401d-b0aa-6676e32f37da service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:17.312838 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c835839-f52e-4282-9a62-0d576349e8a5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '857eb7657e7d45938baca17f91a16901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:17.313438 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c835839-f52e-4282-9a62-0d576349e8a5 service swift] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:17.313770 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c835839-f52e-4282-9a62-0d576349e8a5 service swift] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:17.314045 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c835839-f52e-4282-9a62-0d576349e8a5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:17.324080 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c835839-f52e-4282-9a62-0d576349e8a5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:17.325903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c835839-f52e-4282-9a62-0d576349e8a5 service swift] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:17.334535 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1294/2614] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Sep 5 12:29:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 12:29:17.671949 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379f2e1-c982-4ebc-88a8-c9effc6d1096 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:17.672236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379f2e1-c982-4ebc-88a8-c9effc6d1096 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:17.672672 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379f2e1-c982-4ebc-88a8-c9effc6d1096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:17.733736 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-a379f2e1-c982-4ebc-88a8-c9effc6d1096 None None] MFA Rules not processed for user `8c5ae98336c441118d01f46a50f732d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:17.735184 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a379f2e1-c982-4ebc-88a8-c9effc6d1096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:17.760817 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1321/2615] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Sep 5 12:29:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:17.771935 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c835839-f52e-4282-9a62-0d576349e8a5 service swift] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:17.789680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c835839-f52e-4282-9a62-0d576349e8a5 service swift] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:17.807856 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17756b21-b910-4df6-b4fe-82650fcc7467 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '857eb7657e7d45938baca17f91a16901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:17.808889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17756b21-b910-4df6-b4fe-82650fcc7467 service swift] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:17.810144 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17756b21-b910-4df6-b4fe-82650fcc7467 service swift] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:17.812764 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17756b21-b910-4df6-b4fe-82650fcc7467 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:17.831690 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17756b21-b910-4df6-b4fe-82650fcc7467 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:17.834392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17756b21-b910-4df6-b4fe-82650fcc7467 service swift] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:17.865575 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1295/2616] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Sep 5 12:29:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 05 12:29:27.707930 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=47 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:27.844397 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=56 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:34.477060 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41b597-1c1d-4245-b76b-90cfa5a78571 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:34.477619 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41b597-1c1d-4245-b76b-90cfa5a78571 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:34.477619 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41b597-1c1d-4245-b76b-90cfa5a78571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:34.498691 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-bf41b597-1c1d-4245-b76b-90cfa5a78571 None None] MFA Rules not processed for user `8c5ae98336c441118d01f46a50f732d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:34.499944 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-bf41b597-1c1d-4245-b76b-90cfa5a78571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:34.507507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1322/2617] 10.222.0.16 () {58 vars in 967 bytes} [Mon Sep 5 12:29:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:34.520242 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17756b21-b910-4df6-b4fe-82650fcc7467 service swift] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:34.532295 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17756b21-b910-4df6-b4fe-82650fcc7467 service swift] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:34.537749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c9690-cfba-4559-ae6f-f7dbcd3fb1e7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '857eb7657e7d45938baca17f91a16901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:34.541192 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c9690-cfba-4559-ae6f-f7dbcd3fb1e7 service swift] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:34.541596 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c9690-cfba-4559-ae6f-f7dbcd3fb1e7 service swift] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:34.541969 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c9690-cfba-4559-ae6f-f7dbcd3fb1e7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:34.551652 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c9690-cfba-4559-ae6f-f7dbcd3fb1e7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:34.553360 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c9690-cfba-4559-ae6f-f7dbcd3fb1e7 service swift] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:34.569582 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1296/2618] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Sep 5 12:29:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 12:29:36.733877 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf41b597-1c1d-4245-b76b-90cfa5a78571 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:36.747685 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf41b597-1c1d-4245-b76b-90cfa5a78571 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:36.755713 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f024cf50-93c9-433c-bde4-bb63d4f4b91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:36.756385 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024cf50-93c9-433c-bde4-bb63d4f4b91a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:36.756735 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024cf50-93c9-433c-bde4-bb63d4f4b91a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:36.757079 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024cf50-93c9-433c-bde4-bb63d4f4b91a admin admin] PATH_INFO: `/v3/users/51ad1f400be2465dbfb4a14698247496` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:36.758305 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f024cf50-93c9-433c-bde4-bb63d4f4b91a admin admin] RBAC: Authorizing `identity:delete_user(user_id=51ad1f400be2465dbfb4a14698247496)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:36.760361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f024cf50-93c9-433c-bde4-bb63d4f4b91a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:36.812725 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-f024cf50-93c9-433c-bde4-bb63d4f4b91a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51ad1f400be2465dbfb4a14698247496'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:36.819946 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-f024cf50-93c9-433c-bde4-bb63d4f4b91a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51ad1f400be2465dbfb4a14698247496'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:36.820646 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-f024cf50-93c9-433c-bde4-bb63d4f4b91a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51ad1f400be2465dbfb4a14698247496'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:36.821276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-f024cf50-93c9-433c-bde4-bb63d4f4b91a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51ad1f400be2465dbfb4a14698247496'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:36.833842 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-f024cf50-93c9-433c-bde4-bb63d4f4b91a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51ad1f400be2465dbfb4a14698247496'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:36.846387 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1323/2619] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:29:36 2022] DELETE /identity/v3/users/51ad1f400be2465dbfb4a14698247496 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:37.019453 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c9690-cfba-4559-ae6f-f7dbcd3fb1e7 service swift] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:37.030958 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c9690-cfba-4559-ae6f-f7dbcd3fb1e7 service swift] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:37.093053 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee72f3-f441-4912-9fa9-1e6d73a187f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:37.093683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee72f3-f441-4912-9fa9-1e6d73a187f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:37.093956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee72f3-f441-4912-9fa9-1e6d73a187f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:37.094229 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee72f3-f441-4912-9fa9-1e6d73a187f7 admin admin] PATH_INFO: `/v3/projects/1645a9dd82b14c25892cec9be947f310` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:37.095333 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ee72f3-f441-4912-9fa9-1e6d73a187f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1645a9dd82b14c25892cec9be947f310)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:37.097301 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ee72f3-f441-4912-9fa9-1e6d73a187f7 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:37.165072 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b5ee72f3-f441-4912-9fa9-1e6d73a187f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1645a9dd82b14c25892cec9be947f310', 'request_id': 'req-b5ee72f3-f441-4912-9fa9-1e6d73a187f7'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:37.173392 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b5ee72f3-f441-4912-9fa9-1e6d73a187f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1645a9dd82b14c25892cec9be947f310', 'request_id': 'req-b5ee72f3-f441-4912-9fa9-1e6d73a187f7'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:37.181937 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b5ee72f3-f441-4912-9fa9-1e6d73a187f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1645a9dd82b14c25892cec9be947f310', 'request_id': 'req-b5ee72f3-f441-4912-9fa9-1e6d73a187f7'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:37.197266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b5ee72f3-f441-4912-9fa9-1e6d73a187f7 admin admin] The token cache is being invalidate because project 1645a9dd82b14c25892cec9be947f310 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:29:37.197574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b5ee72f3-f441-4912-9fa9-1e6d73a187f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:37.198506 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b5ee72f3-f441-4912-9fa9-1e6d73a187f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:37.211652 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1297/2620] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:29:37 2022] DELETE /identity/v3/projects/1645a9dd82b14c25892cec9be947f310 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:38.708376 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f024cf50-93c9-433c-bde4-bb63d4f4b91a admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:38.710539 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f024cf50-93c9-433c-bde4-bb63d4f4b91a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:38.788681 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f024cf50-93c9-433c-bde4-bb63d4f4b91a admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:38.870156 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778e5d4-7a6d-4514-ae2e-8f5212dd1070 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:38.876005 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778e5d4-7a6d-4514-ae2e-8f5212dd1070 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:38.876379 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778e5d4-7a6d-4514-ae2e-8f5212dd1070 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:38.876739 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778e5d4-7a6d-4514-ae2e-8f5212dd1070 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:38.878819 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8778e5d4-7a6d-4514-ae2e-8f5212dd1070 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:38.939584 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:38.974107 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8778e5d4-7a6d-4514-ae2e-8f5212dd1070 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:38.975655 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8778e5d4-7a6d-4514-ae2e-8f5212dd1070 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:38.986515 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1324/2621] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:41.082920 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256efc4-e697-461c-a1e5-358053783a32 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:41.083345 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256efc4-e697-461c-a1e5-358053783a32 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:41.083904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256efc4-e697-461c-a1e5-358053783a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:41.105703 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-2256efc4-e697-461c-a1e5-358053783a32 None None] MFA Rules not processed for user `8c5ae98336c441118d01f46a50f732d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:41.106976 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-2256efc4-e697-461c-a1e5-358053783a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:41.199714 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1298/2622] 10.222.0.16 () {58 vars in 967 bytes} [Mon Sep 5 12:29:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:41.207085 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778e5d4-7a6d-4514-ae2e-8f5212dd1070 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:41.210125 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8778e5d4-7a6d-4514-ae2e-8f5212dd1070 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:41.351468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778e5d4-7a6d-4514-ae2e-8f5212dd1070 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:41.357373 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc99ba-77cc-49a4-b425-70762c832e50 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '857eb7657e7d45938baca17f91a16901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:41.358015 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc99ba-77cc-49a4-b425-70762c832e50 service swift] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:41.358290 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc99ba-77cc-49a4-b425-70762c832e50 service swift] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:41.358567 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc99ba-77cc-49a4-b425-70762c832e50 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:41.367665 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dc99ba-77cc-49a4-b425-70762c832e50 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:41.370948 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dc99ba-77cc-49a4-b425-70762c832e50 service swift] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:41.381376 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1325/2623] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Sep 5 12:29:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 05 12:29:42.548323 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256efc4-e697-461c-a1e5-358053783a32 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:42.549796 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-2256efc4-e697-461c-a1e5-358053783a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:42.577864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:42.635768 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256efc4-e697-461c-a1e5-358053783a32 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:42.641421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b63719-5412-4c9b-b31b-be58c977f38d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524bcfffcdfa4790885e0d4cbd66fa80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:42.642025 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b63719-5412-4c9b-b31b-be58c977f38d service placement] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:42.642305 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b63719-5412-4c9b-b31b-be58c977f38d service placement] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:42.642578 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b63719-5412-4c9b-b31b-be58c977f38d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:42.644331 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-26b63719-5412-4c9b-b31b-be58c977f38d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:42.657060 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b63719-5412-4c9b-b31b-be58c977f38d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:42.658650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b63719-5412-4c9b-b31b-be58c977f38d service placement] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:42.668760 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1299/2624] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:46.105665 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc99ba-77cc-49a4-b425-70762c832e50 service swift] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:46.107754 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-82dc99ba-77cc-49a4-b425-70762c832e50 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:46.123299 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc99ba-77cc-49a4-b425-70762c832e50 service swift] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:46.128014 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c47d1e2-6f55-4e6c-98cd-dbbe39af24e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:46.128635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47d1e2-6f55-4e6c-98cd-dbbe39af24e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:46.128916 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47d1e2-6f55-4e6c-98cd-dbbe39af24e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:46.129191 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47d1e2-6f55-4e6c-98cd-dbbe39af24e0 admin admin] PATH_INFO: `/v3/users/8ff56403efde4509bc830967a1c25ccc` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:46.130326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c47d1e2-6f55-4e6c-98cd-dbbe39af24e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ff56403efde4509bc830967a1c25ccc)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:46.132281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c47d1e2-6f55-4e6c-98cd-dbbe39af24e0 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:46.179014 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4c47d1e2-6f55-4e6c-98cd-dbbe39af24e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ff56403efde4509bc830967a1c25ccc'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:46.186725 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4c47d1e2-6f55-4e6c-98cd-dbbe39af24e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ff56403efde4509bc830967a1c25ccc'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:46.187430 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4c47d1e2-6f55-4e6c-98cd-dbbe39af24e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ff56403efde4509bc830967a1c25ccc'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:46.188077 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4c47d1e2-6f55-4e6c-98cd-dbbe39af24e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ff56403efde4509bc830967a1c25ccc'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:46.200237 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4c47d1e2-6f55-4e6c-98cd-dbbe39af24e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ff56403efde4509bc830967a1c25ccc'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:46.216180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1326/2625] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:29:46 2022] DELETE /identity/v3/users/8ff56403efde4509bc830967a1c25ccc => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:46.394189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b63719-5412-4c9b-b31b-be58c977f38d service placement] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:46.405791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b63719-5412-4c9b-b31b-be58c977f38d service placement] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:46.464190 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3db563-d6de-4e93-b001-01f2bc001261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:46.464447 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3db563-d6de-4e93-b001-01f2bc001261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:46.464730 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3db563-d6de-4e93-b001-01f2bc001261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:46.465003 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3db563-d6de-4e93-b001-01f2bc001261 admin admin] PATH_INFO: `/v3/projects/88e8ed647b3845119144655e27322ae1` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:46.466781 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3db563-d6de-4e93-b001-01f2bc001261 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88e8ed647b3845119144655e27322ae1)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:46.468724 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3db563-d6de-4e93-b001-01f2bc001261 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:46.518386 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-5c3db563-d6de-4e93-b001-01f2bc001261 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88e8ed647b3845119144655e27322ae1', 'request_id': 'req-5c3db563-d6de-4e93-b001-01f2bc001261'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:46.525526 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-5c3db563-d6de-4e93-b001-01f2bc001261 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88e8ed647b3845119144655e27322ae1', 'request_id': 'req-5c3db563-d6de-4e93-b001-01f2bc001261'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:46.536339 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-5c3db563-d6de-4e93-b001-01f2bc001261 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88e8ed647b3845119144655e27322ae1', 'request_id': 'req-5c3db563-d6de-4e93-b001-01f2bc001261'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:46.548592 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-5c3db563-d6de-4e93-b001-01f2bc001261 admin admin] The token cache is being invalidate because project 88e8ed647b3845119144655e27322ae1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:29:46.548884 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-5c3db563-d6de-4e93-b001-01f2bc001261 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:46.549534 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-5c3db563-d6de-4e93-b001-01f2bc001261 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:46.551018 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1300/2626] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:29:46 2022] DELETE /identity/v3/projects/88e8ed647b3845119144655e27322ae1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:46.555022 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c47d1e2-6f55-4e6c-98cd-dbbe39af24e0 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:46.556468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4c47d1e2-6f55-4e6c-98cd-dbbe39af24e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:46.570075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c47d1e2-6f55-4e6c-98cd-dbbe39af24e0 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:46.589789 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:46.633647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c434e594-3116-4005-94a2-7ed238b5c0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:46.634293 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434e594-3116-4005-94a2-7ed238b5c0ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:46.634575 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434e594-3116-4005-94a2-7ed238b5c0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:46.634849 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434e594-3116-4005-94a2-7ed238b5c0ba admin admin] PATH_INFO: `/v3/users/d9f4cca8aa5443c18c94bca796813c4f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:46.636158 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c434e594-3116-4005-94a2-7ed238b5c0ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9f4cca8aa5443c18c94bca796813c4f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:46.638525 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c434e594-3116-4005-94a2-7ed238b5c0ba admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:46.687473 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c434e594-3116-4005-94a2-7ed238b5c0ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9f4cca8aa5443c18c94bca796813c4f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:46.694799 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c434e594-3116-4005-94a2-7ed238b5c0ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9f4cca8aa5443c18c94bca796813c4f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:46.695619 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c434e594-3116-4005-94a2-7ed238b5c0ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9f4cca8aa5443c18c94bca796813c4f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:46.696314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c434e594-3116-4005-94a2-7ed238b5c0ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9f4cca8aa5443c18c94bca796813c4f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:46.709814 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c434e594-3116-4005-94a2-7ed238b5c0ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9f4cca8aa5443c18c94bca796813c4f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:46.723697 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1327/2627] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:29:46 2022] DELETE /identity/v3/users/d9f4cca8aa5443c18c94bca796813c4f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:46.936428 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3db563-d6de-4e93-b001-01f2bc001261 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:46.940235 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-5c3db563-d6de-4e93-b001-01f2bc001261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:46.965263 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3db563-d6de-4e93-b001-01f2bc001261 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:47.018406 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:47.042833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7751e0-943f-4614-971c-d82e3b7aecae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:47.043548 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7751e0-943f-4614-971c-d82e3b7aecae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:47.043822 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7751e0-943f-4614-971c-d82e3b7aecae admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:47.044443 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7751e0-943f-4614-971c-d82e3b7aecae admin admin] PATH_INFO: `/v3/projects/8a77c0c3fc9249ceb708c934a5c77df3` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:47.045889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7751e0-943f-4614-971c-d82e3b7aecae admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a77c0c3fc9249ceb708c934a5c77df3)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:47.048108 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7751e0-943f-4614-971c-d82e3b7aecae admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:47.108119 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3a7751e0-943f-4614-971c-d82e3b7aecae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a77c0c3fc9249ceb708c934a5c77df3', 'request_id': 'req-3a7751e0-943f-4614-971c-d82e3b7aecae'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:47.120253 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3a7751e0-943f-4614-971c-d82e3b7aecae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a77c0c3fc9249ceb708c934a5c77df3', 'request_id': 'req-3a7751e0-943f-4614-971c-d82e3b7aecae'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:47.135735 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3a7751e0-943f-4614-971c-d82e3b7aecae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a77c0c3fc9249ceb708c934a5c77df3', 'request_id': 'req-3a7751e0-943f-4614-971c-d82e3b7aecae'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:47.151167 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3a7751e0-943f-4614-971c-d82e3b7aecae admin admin] The token cache is being invalidate because project 8a77c0c3fc9249ceb708c934a5c77df3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:29:47.151462 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3a7751e0-943f-4614-971c-d82e3b7aecae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:47.152839 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3a7751e0-943f-4614-971c-d82e3b7aecae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:47.154892 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1301/2628] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:29:46 2022] DELETE /identity/v3/projects/8a77c0c3fc9249ceb708c934a5c77df3 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:56.720062 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:57.147250 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=32,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:04.774464 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c434e594-3116-4005-94a2-7ed238b5c0ba admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:04.776936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c434e594-3116-4005-94a2-7ed238b5c0ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:04.867811 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c434e594-3116-4005-94a2-7ed238b5c0ba admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:04.974006 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ee90d-b0e0-44bc-b465-4421e1c6ff69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:04.975580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ee90d-b0e0-44bc-b465-4421e1c6ff69 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:04.976396 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ee90d-b0e0-44bc-b465-4421e1c6ff69 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:04.976877 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ee90d-b0e0-44bc-b465-4421e1c6ff69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:04.979244 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e98ee90d-b0e0-44bc-b465-4421e1c6ff69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:04.997294 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ee90d-b0e0-44bc-b465-4421e1c6ff69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:04.999101 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ee90d-b0e0-44bc-b465-4421e1c6ff69 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:05.010890 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1328/2629] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:10.519066 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7751e0-943f-4614-971c-d82e3b7aecae admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:10.520507 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-3a7751e0-943f-4614-971c-d82e3b7aecae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:10.583478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7751e0-943f-4614-971c-d82e3b7aecae admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:10.587347 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc408e3-187e-4915-95de-c081d0cdd9cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:10.587982 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc408e3-187e-4915-95de-c081d0cdd9cf service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:10.588258 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc408e3-187e-4915-95de-c081d0cdd9cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:10.588529 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc408e3-187e-4915-95de-c081d0cdd9cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:10.590224 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-5bc408e3-187e-4915-95de-c081d0cdd9cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:10.650839 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc408e3-187e-4915-95de-c081d0cdd9cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:10.652496 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc408e3-187e-4915-95de-c081d0cdd9cf service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:10.662841 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1302/2630] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:13.827801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ee90d-b0e0-44bc-b465-4421e1c6ff69 service nova] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:13.829792 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e98ee90d-b0e0-44bc-b465-4421e1c6ff69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:13.843135 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ee90d-b0e0-44bc-b465-4421e1c6ff69 service nova] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:13.847899 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064aad0-7ae6-44ae-b55b-ba9c312a6e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:13.848609 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064aad0-7ae6-44ae-b55b-ba9c312a6e1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:13.848893 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064aad0-7ae6-44ae-b55b-ba9c312a6e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:13.849166 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064aad0-7ae6-44ae-b55b-ba9c312a6e1e admin admin] PATH_INFO: `/v3/users/45bd61574c6c4ebb9d6fe0c87228065f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:13.866518 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b064aad0-7ae6-44ae-b55b-ba9c312a6e1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=45bd61574c6c4ebb9d6fe0c87228065f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:13.868560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b064aad0-7ae6-44ae-b55b-ba9c312a6e1e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:13.914887 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b064aad0-7ae6-44ae-b55b-ba9c312a6e1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45bd61574c6c4ebb9d6fe0c87228065f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:13.921842 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b064aad0-7ae6-44ae-b55b-ba9c312a6e1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45bd61574c6c4ebb9d6fe0c87228065f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:13.922531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b064aad0-7ae6-44ae-b55b-ba9c312a6e1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45bd61574c6c4ebb9d6fe0c87228065f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:13.923105 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b064aad0-7ae6-44ae-b55b-ba9c312a6e1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45bd61574c6c4ebb9d6fe0c87228065f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:13.930663 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:13.935715 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b064aad0-7ae6-44ae-b55b-ba9c312a6e1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45bd61574c6c4ebb9d6fe0c87228065f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:13.948545 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1329/2631] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:13 2022] DELETE /identity/v3/users/45bd61574c6c4ebb9d6fe0c87228065f => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:14.116774 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc408e3-187e-4915-95de-c081d0cdd9cf service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:14.128926 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc408e3-187e-4915-95de-c081d0cdd9cf service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:14.189696 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9aa66-7907-4c8f-a8a6-6fe810c39340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:14.190320 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9aa66-7907-4c8f-a8a6-6fe810c39340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:14.190596 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9aa66-7907-4c8f-a8a6-6fe810c39340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:14.190867 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9aa66-7907-4c8f-a8a6-6fe810c39340 admin admin] PATH_INFO: `/v3/projects/8f21d577dc6d44a7b05ee0a0232bed6c` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:14.201431 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe9aa66-7907-4c8f-a8a6-6fe810c39340 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f21d577dc6d44a7b05ee0a0232bed6c)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:14.203399 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe9aa66-7907-4c8f-a8a6-6fe810c39340 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:14.246127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:14.251002 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-dfe9aa66-7907-4c8f-a8a6-6fe810c39340 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f21d577dc6d44a7b05ee0a0232bed6c', 'request_id': 'req-dfe9aa66-7907-4c8f-a8a6-6fe810c39340'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:14.258879 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-dfe9aa66-7907-4c8f-a8a6-6fe810c39340 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f21d577dc6d44a7b05ee0a0232bed6c', 'request_id': 'req-dfe9aa66-7907-4c8f-a8a6-6fe810c39340'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:14.264655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-dfe9aa66-7907-4c8f-a8a6-6fe810c39340 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f21d577dc6d44a7b05ee0a0232bed6c', 'request_id': 'req-dfe9aa66-7907-4c8f-a8a6-6fe810c39340'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:14.276537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-dfe9aa66-7907-4c8f-a8a6-6fe810c39340 admin admin] The token cache is being invalidate because project 8f21d577dc6d44a7b05ee0a0232bed6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:14.276822 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-dfe9aa66-7907-4c8f-a8a6-6fe810c39340 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:14.277453 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-dfe9aa66-7907-4c8f-a8a6-6fe810c39340 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:14.278923 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1303/2632] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:14 2022] DELETE /identity/v3/projects/8f21d577dc6d44a7b05ee0a0232bed6c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:14.284183 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354cec0-8759-4387-aafb-be14cfc0fba8 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:14.284468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354cec0-8759-4387-aafb-be14cfc0fba8 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:14.284743 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354cec0-8759-4387-aafb-be14cfc0fba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:14.304912 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-2354cec0-8759-4387-aafb-be14cfc0fba8 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:14.306125 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2354cec0-8759-4387-aafb-be14cfc0fba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:14.364949 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1330/2633] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:30:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:14.373301 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebc821-a93f-4144-9a1a-ff1dc323db75 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:14.373587 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebc821-a93f-4144-9a1a-ff1dc323db75 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:14.373862 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebc821-a93f-4144-9a1a-ff1dc323db75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:14.395358 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-7eebc821-a93f-4144-9a1a-ff1dc323db75 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:14.396930 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-7eebc821-a93f-4144-9a1a-ff1dc323db75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:14.407017 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1304/2634] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:30:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:14.411722 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354cec0-8759-4387-aafb-be14cfc0fba8 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:14.422277 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354cec0-8759-4387-aafb-be14cfc0fba8 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:14.426873 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e39d8e-75f0-4740-aefd-98c877eedcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:14.427468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e39d8e-75f0-4740-aefd-98c877eedcb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:14.427786 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e39d8e-75f0-4740-aefd-98c877eedcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:14.428055 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e39d8e-75f0-4740-aefd-98c877eedcb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:14.428720 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e39d8e-75f0-4740-aefd-98c877eedcb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:30:14.429017 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e39d8e-75f0-4740-aefd-98c877eedcb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:14.430629 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e39d8e-75f0-4740-aefd-98c877eedcb7 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:14.442752 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1331/2635] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:30:14.447453 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eebc821-a93f-4144-9a1a-ff1dc323db75 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:14.456831 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eebc821-a93f-4144-9a1a-ff1dc323db75 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:14.461977 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6f08c-c159-4b56-870d-28858f8a33f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:14.462678 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6f08c-c159-4b56-870d-28858f8a33f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:14.462936 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6f08c-c159-4b56-870d-28858f8a33f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:14.463201 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6f08c-c159-4b56-870d-28858f8a33f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:14.463851 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d6f08c-c159-4b56-870d-28858f8a33f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:14.465916 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d6f08c-c159-4b56-870d-28858f8a33f6 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:14.503659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1305/2636] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:30:14 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:14.507904 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e39d8e-75f0-4740-aefd-98c877eedcb7 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:14.517847 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e39d8e-75f0-4740-aefd-98c877eedcb7 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:14.574020 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f78d85-917d-422e-b4af-21d71de4be75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:14.574647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f78d85-917d-422e-b4af-21d71de4be75 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:14.574928 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f78d85-917d-422e-b4af-21d71de4be75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:14.575199 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f78d85-917d-422e-b4af-21d71de4be75 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:14.575877 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f78d85-917d-422e-b4af-21d71de4be75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:14.577777 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f78d85-917d-422e-b4af-21d71de4be75 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:14.599828 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1332/2637] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:30:14 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:14.603793 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6f08c-c159-4b56-870d-28858f8a33f6 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:14.614584 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6f08c-c159-4b56-870d-28858f8a33f6 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:14.619287 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01629137-bf80-4364-8f8a-8e3237fb5154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:14.619849 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01629137-bf80-4364-8f8a-8e3237fb5154 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:14.620121 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01629137-bf80-4364-8f8a-8e3237fb5154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:14.620390 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01629137-bf80-4364-8f8a-8e3237fb5154 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:14.621023 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01629137-bf80-4364-8f8a-8e3237fb5154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:14.622742 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01629137-bf80-4364-8f8a-8e3237fb5154 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:14.632475 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1306/2638] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:14.636522 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f78d85-917d-422e-b4af-21d71de4be75 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:14.646476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f78d85-917d-422e-b4af-21d71de4be75 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:14.651414 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae63857-61ff-4908-827e-6ffba8b8d137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:14.652043 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae63857-61ff-4908-827e-6ffba8b8d137 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:14.652324 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae63857-61ff-4908-827e-6ffba8b8d137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:14.652604 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae63857-61ff-4908-827e-6ffba8b8d137 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:14.653236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae63857-61ff-4908-827e-6ffba8b8d137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:14.654821 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae63857-61ff-4908-827e-6ffba8b8d137 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:14.666032 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1333/2639] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:14.670282 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01629137-bf80-4364-8f8a-8e3237fb5154 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:14.681084 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01629137-bf80-4364-8f8a-8e3237fb5154 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:14.685734 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d1c8d-5e27-4ae6-81e4-aa6e9c840f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:14.686371 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d1c8d-5e27-4ae6-81e4-aa6e9c840f91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:14.686652 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d1c8d-5e27-4ae6-81e4-aa6e9c840f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:14.686930 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d1c8d-5e27-4ae6-81e4-aa6e9c840f91 admin admin] PATH_INFO: `/v3/projects/0d3972b5b8b24e999998d2616531f9e7/users/ee580cfc10dd4536b0709e65e3d23d4d/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:14.703130 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d1c8d-5e27-4ae6-81e4-aa6e9c840f91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d3972b5b8b24e999998d2616531f9e7, user_id=ee580cfc10dd4536b0709e65e3d23d4d, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:14.705842 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d1c8d-5e27-4ae6-81e4-aa6e9c840f91 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:14.719708 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1307/2640] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:14 2022] PUT /identity/v3/projects/0d3972b5b8b24e999998d2616531f9e7/users/ee580cfc10dd4536b0709e65e3d23d4d/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:14.724828 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508219d-ec55-44e6-8d09-91b461b8491c None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:14.725145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508219d-ec55-44e6-8d09-91b461b8491c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:14.725426 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508219d-ec55-44e6-8d09-91b461b8491c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:14.745149 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-8508219d-ec55-44e6-8d09-91b461b8491c None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:14.746360 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8508219d-ec55-44e6-8d09-91b461b8491c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:14.791738 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:14.807885 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1334/2641] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:30:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:30:14.817741 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d1c8d-5e27-4ae6-81e4-aa6e9c840f91 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:14.819416 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-d93d1c8d-5e27-4ae6-81e4-aa6e9c840f91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:14.875821 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d1c8d-5e27-4ae6-81e4-aa6e9c840f91 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:14.879273 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208e3ae-35c8-4b0f-b3bc-35d425f2eb4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:14.879887 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208e3ae-35c8-4b0f-b3bc-35d425f2eb4b service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:14.880165 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208e3ae-35c8-4b0f-b3bc-35d425f2eb4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:14.880432 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208e3ae-35c8-4b0f-b3bc-35d425f2eb4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:14.890183 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208e3ae-35c8-4b0f-b3bc-35d425f2eb4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:14.891622 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208e3ae-35c8-4b0f-b3bc-35d425f2eb4b service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:14.900911 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1308/2642] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:19.018884 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc21697-bfb2-4355-bce9-4e983753c21a None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:19.019434 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc21697-bfb2-4355-bce9-4e983753c21a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:19.019568 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc21697-bfb2-4355-bce9-4e983753c21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:19.054307 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-1dc21697-bfb2-4355-bce9-4e983753c21a None None] MFA Rules not processed for user `ee580cfc10dd4536b0709e65e3d23d4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:19.055805 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1dc21697-bfb2-4355-bce9-4e983753c21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:19.132018 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1335/2643] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:30:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:19.146658 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208e3ae-35c8-4b0f-b3bc-35d425f2eb4b service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:19.148897 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-4208e3ae-35c8-4b0f-b3bc-35d425f2eb4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:19.213642 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:19.235281 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208e3ae-35c8-4b0f-b3bc-35d425f2eb4b service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:19.250682 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5405a-9f8a-4dfc-8635-c6fc9bbe4a4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:19.251385 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5405a-9f8a-4dfc-8635-c6fc9bbe4a4b service nova] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:19.251710 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5405a-9f8a-4dfc-8635-c6fc9bbe4a4b service nova] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:19.251984 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5405a-9f8a-4dfc-8635-c6fc9bbe4a4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:19.261671 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5405a-9f8a-4dfc-8635-c6fc9bbe4a4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:19.263782 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5405a-9f8a-4dfc-8635-c6fc9bbe4a4b service nova] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:19.274436 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1309/2644] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:20.646596 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2251832-15f6-486d-9da2-bc84a5a32c38 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:20.647183 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2251832-15f6-486d-9da2-bc84a5a32c38 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:20.647183 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2251832-15f6-486d-9da2-bc84a5a32c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:20.669292 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-e2251832-15f6-486d-9da2-bc84a5a32c38 None None] MFA Rules not processed for user `ee580cfc10dd4536b0709e65e3d23d4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:20.670645 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e2251832-15f6-486d-9da2-bc84a5a32c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:20.681645 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1336/2645] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:30:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:21.038127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5405a-9f8a-4dfc-8635-c6fc9bbe4a4b service nova] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:21.040990 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-46b5405a-9f8a-4dfc-8635-c6fc9bbe4a4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:21.056078 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5405a-9f8a-4dfc-8635-c6fc9bbe4a4b service nova] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:21.060266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5baa9-9a3e-492d-ad43-eace74f65435 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:21.060872 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5baa9-9a3e-492d-ad43-eace74f65435 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:21.061152 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5baa9-9a3e-492d-ad43-eace74f65435 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:21.061425 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5baa9-9a3e-492d-ad43-eace74f65435 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:21.063857 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-c9e5baa9-9a3e-492d-ad43-eace74f65435 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:21.075166 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5baa9-9a3e-492d-ad43-eace74f65435 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:21.076916 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5baa9-9a3e-492d-ad43-eace74f65435 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:21.085951 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1310/2646] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:21.502189 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2251832-15f6-486d-9da2-bc84a5a32c38 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:21.503968 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e2251832-15f6-486d-9da2-bc84a5a32c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:21.596395 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2251832-15f6-486d-9da2-bc84a5a32c38 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:21.602930 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ab359-1107-403d-8ea8-375ed941c2fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524bcfffcdfa4790885e0d4cbd66fa80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:21.603596 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ab359-1107-403d-8ea8-375ed941c2fa service placement] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:21.603988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ab359-1107-403d-8ea8-375ed941c2fa service placement] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:21.604218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ab359-1107-403d-8ea8-375ed941c2fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:21.606398 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d67ab359-1107-403d-8ea8-375ed941c2fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:21.621460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67ab359-1107-403d-8ea8-375ed941c2fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:21.623381 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67ab359-1107-403d-8ea8-375ed941c2fa service placement] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:21.635208 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1337/2647] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:31.039511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5baa9-9a3e-492d-ad43-eace74f65435 service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:31.053512 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5baa9-9a3e-492d-ad43-eace74f65435 service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:31.059015 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618d49a-c911-4e8b-b7d1-3a79b7157ae6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:31.059652 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618d49a-c911-4e8b-b7d1-3a79b7157ae6 service nova] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:31.059932 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618d49a-c911-4e8b-b7d1-3a79b7157ae6 service nova] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:31.060281 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618d49a-c911-4e8b-b7d1-3a79b7157ae6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:31.061937 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-0618d49a-c911-4e8b-b7d1-3a79b7157ae6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:31.075329 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0618d49a-c911-4e8b-b7d1-3a79b7157ae6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:31.076939 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0618d49a-c911-4e8b-b7d1-3a79b7157ae6 service nova] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:31.088466 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1311/2648] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:31.630168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=71 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:41.083969 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=18 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:48.114324 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ab359-1107-403d-8ea8-375ed941c2fa service placement] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:48.124040 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ab359-1107-403d-8ea8-375ed941c2fa service placement] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:48.128112 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24d9cf-b31d-4024-8e02-a446b116ebca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:48.128711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24d9cf-b31d-4024-8e02-a446b116ebca service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:48.128989 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24d9cf-b31d-4024-8e02-a446b116ebca service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:48.129265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24d9cf-b31d-4024-8e02-a446b116ebca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:48.130981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-cc24d9cf-b31d-4024-8e02-a446b116ebca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:48.143625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc24d9cf-b31d-4024-8e02-a446b116ebca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:48.145030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc24d9cf-b31d-4024-8e02-a446b116ebca service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:48.155090 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1338/2649] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:51.338625 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618d49a-c911-4e8b-b7d1-3a79b7157ae6 service nova] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:51.340230 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-0618d49a-c911-4e8b-b7d1-3a79b7157ae6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:51.363732 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618d49a-c911-4e8b-b7d1-3a79b7157ae6 service nova] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:51.373597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d0e61-8ef1-4a36-8cc7-21d0adda5d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:51.373597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d0e61-8ef1-4a36-8cc7-21d0adda5d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:51.373597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d0e61-8ef1-4a36-8cc7-21d0adda5d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:51.373597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d0e61-8ef1-4a36-8cc7-21d0adda5d69 admin admin] PATH_INFO: `/v3/users/83d3b245c185424a9b6e5d69d691f031` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:51.375030 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d0e61-8ef1-4a36-8cc7-21d0adda5d69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83d3b245c185424a9b6e5d69d691f031)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:51.377878 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d0e61-8ef1-4a36-8cc7-21d0adda5d69 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:51.427433 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-6f5d0e61-8ef1-4a36-8cc7-21d0adda5d69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83d3b245c185424a9b6e5d69d691f031'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:51.435349 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-6f5d0e61-8ef1-4a36-8cc7-21d0adda5d69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83d3b245c185424a9b6e5d69d691f031'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:51.436350 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-6f5d0e61-8ef1-4a36-8cc7-21d0adda5d69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83d3b245c185424a9b6e5d69d691f031'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:51.448341 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-6f5d0e61-8ef1-4a36-8cc7-21d0adda5d69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83d3b245c185424a9b6e5d69d691f031'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:51.455617 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-6f5d0e61-8ef1-4a36-8cc7-21d0adda5d69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83d3b245c185424a9b6e5d69d691f031'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:51.461347 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1312/2650] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:51 2022] DELETE /identity/v3/users/83d3b245c185424a9b6e5d69d691f031 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:51.640957 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24d9cf-b31d-4024-8e02-a446b116ebca service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:51.654958 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24d9cf-b31d-4024-8e02-a446b116ebca service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:51.714409 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c9940-044e-4272-a4f6-36e9dec70478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:51.715066 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c9940-044e-4272-a4f6-36e9dec70478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:51.715350 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c9940-044e-4272-a4f6-36e9dec70478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:51.715666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c9940-044e-4272-a4f6-36e9dec70478 admin admin] PATH_INFO: `/v3/projects/ca5b70833b074d199437d3f10e511306` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:51.716797 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c9940-044e-4272-a4f6-36e9dec70478 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca5b70833b074d199437d3f10e511306)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:51.718682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c9940-044e-4272-a4f6-36e9dec70478 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:51.765883 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ea5c9940-044e-4272-a4f6-36e9dec70478 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca5b70833b074d199437d3f10e511306', 'request_id': 'req-ea5c9940-044e-4272-a4f6-36e9dec70478'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:51.772253 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ea5c9940-044e-4272-a4f6-36e9dec70478 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca5b70833b074d199437d3f10e511306', 'request_id': 'req-ea5c9940-044e-4272-a4f6-36e9dec70478'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:51.778647 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ea5c9940-044e-4272-a4f6-36e9dec70478 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca5b70833b074d199437d3f10e511306', 'request_id': 'req-ea5c9940-044e-4272-a4f6-36e9dec70478'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:51.792253 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ea5c9940-044e-4272-a4f6-36e9dec70478 admin admin] The token cache is being invalidate because project ca5b70833b074d199437d3f10e511306 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:51.792545 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ea5c9940-044e-4272-a4f6-36e9dec70478 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:51.793180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ea5c9940-044e-4272-a4f6-36e9dec70478 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:51.794720 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1339/2651] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:51 2022] DELETE /identity/v3/projects/ca5b70833b074d199437d3f10e511306 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:51.799839 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf353fd-f58d-47ab-8499-613046bb83bf None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:51.800122 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf353fd-f58d-47ab-8499-613046bb83bf None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:51.800397 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf353fd-f58d-47ab-8499-613046bb83bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:51.820152 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-edf353fd-f58d-47ab-8499-613046bb83bf None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:51.821352 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-edf353fd-f58d-47ab-8499-613046bb83bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:51.880153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1313/2652] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:51.888301 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f95467-4b12-4d7c-9c65-d14954072447 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:51.888520 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f95467-4b12-4d7c-9c65-d14954072447 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:51.888790 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f95467-4b12-4d7c-9c65-d14954072447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:51.909732 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-58f95467-4b12-4d7c-9c65-d14954072447 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:51.910911 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-58f95467-4b12-4d7c-9c65-d14954072447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:51.919106 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1340/2653] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:51.923496 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf353fd-f58d-47ab-8499-613046bb83bf None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:51.933167 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf353fd-f58d-47ab-8499-613046bb83bf None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:51.937777 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c408abf-c332-4217-82c8-303efcd63040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:51.938388 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c408abf-c332-4217-82c8-303efcd63040 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:51.938643 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c408abf-c332-4217-82c8-303efcd63040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:51.938911 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c408abf-c332-4217-82c8-303efcd63040 admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:51.939583 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c408abf-c332-4217-82c8-303efcd63040 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:30:51.939891 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c408abf-c332-4217-82c8-303efcd63040 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:51.941489 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c408abf-c332-4217-82c8-303efcd63040 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:51.954593 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1314/2654] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:30:51.958823 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f95467-4b12-4d7c-9c65-d14954072447 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:51.968470 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f95467-4b12-4d7c-9c65-d14954072447 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:51.973969 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1de7c-b2fa-4a6e-9aba-c6b2d884511f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:51.974578 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1de7c-b2fa-4a6e-9aba-c6b2d884511f admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:51.974876 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1de7c-b2fa-4a6e-9aba-c6b2d884511f admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:51.975144 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1de7c-b2fa-4a6e-9aba-c6b2d884511f admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:51.975813 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b1de7c-b2fa-4a6e-9aba-c6b2d884511f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:51.977744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b1de7c-b2fa-4a6e-9aba-c6b2d884511f admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:52.004719 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1341/2655] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:30:51 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:52.008928 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c408abf-c332-4217-82c8-303efcd63040 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:52.019405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c408abf-c332-4217-82c8-303efcd63040 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:52.075728 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71fc8a3-7f6e-4122-b24f-943bbe6ed269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:52.076318 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fc8a3-7f6e-4122-b24f-943bbe6ed269 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:52.076597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fc8a3-7f6e-4122-b24f-943bbe6ed269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:52.076866 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fc8a3-7f6e-4122-b24f-943bbe6ed269 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:52.077489 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71fc8a3-7f6e-4122-b24f-943bbe6ed269 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:52.079383 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71fc8a3-7f6e-4122-b24f-943bbe6ed269 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:52.087219 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:52.104116 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1315/2656] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:30:52 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:30:52.108416 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1de7c-b2fa-4a6e-9aba-c6b2d884511f admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:52.119469 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1de7c-b2fa-4a6e-9aba-c6b2d884511f admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:52.124116 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4962ffa5-02b0-48ce-9f38-e3c168794fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:52.124708 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4962ffa5-02b0-48ce-9f38-e3c168794fcc admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:52.124989 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4962ffa5-02b0-48ce-9f38-e3c168794fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:52.125293 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4962ffa5-02b0-48ce-9f38-e3c168794fcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:52.125930 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4962ffa5-02b0-48ce-9f38-e3c168794fcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:52.127555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4962ffa5-02b0-48ce-9f38-e3c168794fcc admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:52.137332 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1342/2657] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:52.141320 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71fc8a3-7f6e-4122-b24f-943bbe6ed269 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:52.151404 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71fc8a3-7f6e-4122-b24f-943bbe6ed269 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:52.156008 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d47eda-d7c0-44de-8282-cbf575b4dabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:52.156593 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d47eda-d7c0-44de-8282-cbf575b4dabd admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:52.156873 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d47eda-d7c0-44de-8282-cbf575b4dabd admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:52.157634 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d47eda-d7c0-44de-8282-cbf575b4dabd admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:52.158281 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d47eda-d7c0-44de-8282-cbf575b4dabd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:52.159898 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d47eda-d7c0-44de-8282-cbf575b4dabd admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:52.170225 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1316/2658] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:52.174833 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4962ffa5-02b0-48ce-9f38-e3c168794fcc admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:52.184861 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4962ffa5-02b0-48ce-9f38-e3c168794fcc admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:52.189541 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3b660-5d5c-4275-a999-87dfba3a390a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:52.190212 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3b660-5d5c-4275-a999-87dfba3a390a admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:52.190499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3b660-5d5c-4275-a999-87dfba3a390a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:52.190775 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3b660-5d5c-4275-a999-87dfba3a390a admin admin] PATH_INFO: `/v3/projects/ab2dea97bdd048a291a4b4ee00b5de25/users/e8e37d001dc340e991d16d1605b2f05a/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:52.205415 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce3b660-5d5c-4275-a999-87dfba3a390a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab2dea97bdd048a291a4b4ee00b5de25, user_id=e8e37d001dc340e991d16d1605b2f05a, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:52.208106 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce3b660-5d5c-4275-a999-87dfba3a390a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:52.222122 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1343/2659] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:52 2022] PUT /identity/v3/projects/ab2dea97bdd048a291a4b4ee00b5de25/users/e8e37d001dc340e991d16d1605b2f05a/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:52.233759 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e8aa6-923f-41e5-a012-83951be9489e None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:52.234047 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e8aa6-923f-41e5-a012-83951be9489e None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:52.234324 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e8aa6-923f-41e5-a012-83951be9489e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:52.262317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-6f6e8aa6-923f-41e5-a012-83951be9489e None None] MFA Rules not processed for user `e8e37d001dc340e991d16d1605b2f05a`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:52.263551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-6f6e8aa6-923f-41e5-a012-83951be9489e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:52.328769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1317/2660] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:30:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:52.333312 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3b660-5d5c-4275-a999-87dfba3a390a admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:52.347295 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:52.398009 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3b660-5d5c-4275-a999-87dfba3a390a admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:52.403787 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8cc4fe-4354-4472-bf26-108b0bf5d2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:52.404393 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cc4fe-4354-4472-bf26-108b0bf5d2d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:52.404676 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cc4fe-4354-4472-bf26-108b0bf5d2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:52.404950 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cc4fe-4354-4472-bf26-108b0bf5d2d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:52.405592 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8cc4fe-4354-4472-bf26-108b0bf5d2d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:52.408071 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8cc4fe-4354-4472-bf26-108b0bf5d2d5 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:52.434072 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1344/2661] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:30:52 2022] POST /identity/v3/projects => generated 371 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:52.438366 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e8aa6-923f-41e5-a012-83951be9489e None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:52.449064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e8aa6-923f-41e5-a012-83951be9489e None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:52.505368 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18885350-aaa7-4947-93ed-73d0cc5cfc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:52.506002 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18885350-aaa7-4947-93ed-73d0cc5cfc39 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:52.506282 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18885350-aaa7-4947-93ed-73d0cc5cfc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:52.506553 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18885350-aaa7-4947-93ed-73d0cc5cfc39 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:52.507221 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18885350-aaa7-4947-93ed-73d0cc5cfc39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:52.509223 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18885350-aaa7-4947-93ed-73d0cc5cfc39 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:52.531062 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1318/2662] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:30:52 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:30:52.535648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8cc4fe-4354-4472-bf26-108b0bf5d2d5 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:52.545950 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8cc4fe-4354-4472-bf26-108b0bf5d2d5 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:52.551145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b05eed2-5425-4be9-b559-801d2ec0679f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:52.551874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05eed2-5425-4be9-b559-801d2ec0679f admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:52.552208 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05eed2-5425-4be9-b559-801d2ec0679f admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:52.552526 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05eed2-5425-4be9-b559-801d2ec0679f admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:52.553207 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b05eed2-5425-4be9-b559-801d2ec0679f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:52.554849 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b05eed2-5425-4be9-b559-801d2ec0679f admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:52.566013 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1345/2663] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:52.570386 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18885350-aaa7-4947-93ed-73d0cc5cfc39 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:52.581058 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18885350-aaa7-4947-93ed-73d0cc5cfc39 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:52.585704 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ec2d65-2326-4734-b90c-0b0daf4cf540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:52.586349 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec2d65-2326-4734-b90c-0b0daf4cf540 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:52.586632 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec2d65-2326-4734-b90c-0b0daf4cf540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:52.586913 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec2d65-2326-4734-b90c-0b0daf4cf540 admin admin] PATH_INFO: `/v3/projects/888596d31b7d4725b4e2291872d4adf5/users/2b3d828778d24de99bb6e4e299090972/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:52.602692 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ec2d65-2326-4734-b90c-0b0daf4cf540 admin admin] RBAC: Authorizing `identity:create_grant(project_id=888596d31b7d4725b4e2291872d4adf5, user_id=2b3d828778d24de99bb6e4e299090972, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:52.605578 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ec2d65-2326-4734-b90c-0b0daf4cf540 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:52.619212 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1319/2664] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:52 2022] PUT /identity/v3/projects/888596d31b7d4725b4e2291872d4adf5/users/2b3d828778d24de99bb6e4e299090972/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:52.623344 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b05eed2-5425-4be9-b559-801d2ec0679f admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:52.633404 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b05eed2-5425-4be9-b559-801d2ec0679f admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:52.688555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f099427-2aa9-4b9e-8c59-6aa6925dea60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:52.689199 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f099427-2aa9-4b9e-8c59-6aa6925dea60 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:52.689370 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f099427-2aa9-4b9e-8c59-6aa6925dea60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:52.689645 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f099427-2aa9-4b9e-8c59-6aa6925dea60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:52.690312 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f099427-2aa9-4b9e-8c59-6aa6925dea60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:52.692009 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f099427-2aa9-4b9e-8c59-6aa6925dea60 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:52.701708 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1346/2665] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:52.706573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ec2d65-2326-4734-b90c-0b0daf4cf540 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:52.716708 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ec2d65-2326-4734-b90c-0b0daf4cf540 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:52.721894 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad46b60-0eda-4b40-87ab-cb4517cc444a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:52.722525 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad46b60-0eda-4b40-87ab-cb4517cc444a admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:52.722799 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad46b60-0eda-4b40-87ab-cb4517cc444a admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:52.723072 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad46b60-0eda-4b40-87ab-cb4517cc444a admin admin] PATH_INFO: `/v3/projects/888596d31b7d4725b4e2291872d4adf5/users/2b3d828778d24de99bb6e4e299090972/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:52.724972 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad46b60-0eda-4b40-87ab-cb4517cc444a admin admin] RBAC: Authorizing `identity:create_grant(project_id=888596d31b7d4725b4e2291872d4adf5, user_id=2b3d828778d24de99bb6e4e299090972, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:52.727601 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad46b60-0eda-4b40-87ab-cb4517cc444a admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:52.737538 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1320/2666] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:52 2022] PUT /identity/v3/projects/888596d31b7d4725b4e2291872d4adf5/users/2b3d828778d24de99bb6e4e299090972/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:52.747696 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa45761-d3fa-42de-8237-560aac8427df None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:52.747982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa45761-d3fa-42de-8237-560aac8427df None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:52.748264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa45761-d3fa-42de-8237-560aac8427df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:52.777040 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-0aa45761-d3fa-42de-8237-560aac8427df None None] MFA Rules not processed for user `2b3d828778d24de99bb6e4e299090972`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:52.778274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-0aa45761-d3fa-42de-8237-560aac8427df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:52.848880 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1347/2667] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:30:52 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:52.859867 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad46b60-0eda-4b40-87ab-cb4517cc444a admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:52.861582 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-8ad46b60-0eda-4b40-87ab-cb4517cc444a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:52.884525 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:52.924741 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad46b60-0eda-4b40-87ab-cb4517cc444a admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:52.928705 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae68754-8d37-46a9-98b4-84b93250adb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:52.929284 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae68754-8d37-46a9-98b4-84b93250adb0 service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:52.929558 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae68754-8d37-46a9-98b4-84b93250adb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:52.929826 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae68754-8d37-46a9-98b4-84b93250adb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:52.939667 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae68754-8d37-46a9-98b4-84b93250adb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:52.941248 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae68754-8d37-46a9-98b4-84b93250adb0 service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:52.951388 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1321/2668] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:52 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:53.028625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa45761-d3fa-42de-8237-560aac8427df None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:53.038210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa45761-d3fa-42de-8237-560aac8427df None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:53.041771 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f17199-e94d-463a-b5ec-a78dc221a3a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:53.042359 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f17199-e94d-463a-b5ec-a78dc221a3a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:53.042633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f17199-e94d-463a-b5ec-a78dc221a3a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:53.042903 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f17199-e94d-463a-b5ec-a78dc221a3a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:53.047921 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:53.100011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f17199-e94d-463a-b5ec-a78dc221a3a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:53.101693 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f17199-e94d-463a-b5ec-a78dc221a3a3 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:53.111215 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1348/2669] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:31:02.946429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=17 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:03.106576 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=24 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:37.730042 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae68754-8d37-46a9-98b4-84b93250adb0 service cinder] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:37.793572 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae68754-8d37-46a9-98b4-84b93250adb0 service cinder] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:37.798471 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd88e9-b173-4e45-af4d-da72bcb34cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:37.799113 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd88e9-b173-4e45-af4d-da72bcb34cd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:37.799930 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd88e9-b173-4e45-af4d-da72bcb34cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:37.800255 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd88e9-b173-4e45-af4d-da72bcb34cd3 admin admin] PATH_INFO: `/v3/users/e8e37d001dc340e991d16d1605b2f05a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:37.801420 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd88e9-b173-4e45-af4d-da72bcb34cd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8e37d001dc340e991d16d1605b2f05a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:37.803326 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd88e9-b173-4e45-af4d-da72bcb34cd3 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:37.849685 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-02fd88e9-b173-4e45-af4d-da72bcb34cd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8e37d001dc340e991d16d1605b2f05a'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:37.857056 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-02fd88e9-b173-4e45-af4d-da72bcb34cd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8e37d001dc340e991d16d1605b2f05a'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:37.857666 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-02fd88e9-b173-4e45-af4d-da72bcb34cd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8e37d001dc340e991d16d1605b2f05a'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:37.870614 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-02fd88e9-b173-4e45-af4d-da72bcb34cd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8e37d001dc340e991d16d1605b2f05a'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:37.877288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-02fd88e9-b173-4e45-af4d-da72bcb34cd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8e37d001dc340e991d16d1605b2f05a'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:37.882944 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1322/2670] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:31:37 2022] DELETE /identity/v3/users/e8e37d001dc340e991d16d1605b2f05a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:31:37.887414 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7c47d-6a45-41d3-9d17-9dd011bf1c65 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:37.887938 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7c47d-6a45-41d3-9d17-9dd011bf1c65 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:37.888044 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7c47d-6a45-41d3-9d17-9dd011bf1c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:37.906714 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-19a7c47d-6a45-41d3-9d17-9dd011bf1c65 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:37.907939 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-19a7c47d-6a45-41d3-9d17-9dd011bf1c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:37.965263 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1349/2671] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:31:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:37.975748 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd88e9-b173-4e45-af4d-da72bcb34cd3 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:37.977114 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-02fd88e9-b173-4e45-af4d-da72bcb34cd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:38.032673 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd88e9-b173-4e45-af4d-da72bcb34cd3 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:38.036702 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2210f-a66b-4b62-8a58-aad5f5e31542 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:38.037315 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2210f-a66b-4b62-8a58-aad5f5e31542 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:38.037588 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2210f-a66b-4b62-8a58-aad5f5e31542 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:38.037854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2210f-a66b-4b62-8a58-aad5f5e31542 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:38.046610 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2210f-a66b-4b62-8a58-aad5f5e31542 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:38.048040 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2210f-a66b-4b62-8a58-aad5f5e31542 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:38.057177 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1323/2672] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:31:38.323302 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7c47d-6a45-41d3-9d17-9dd011bf1c65 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:38.324797 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-19a7c47d-6a45-41d3-9d17-9dd011bf1c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:38.336480 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7c47d-6a45-41d3-9d17-9dd011bf1c65 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:38.341454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7e01c-c349-4332-9832-30a7d8f06975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:38.342221 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7e01c-c349-4332-9832-30a7d8f06975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:38.342503 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7e01c-c349-4332-9832-30a7d8f06975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:38.342778 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7e01c-c349-4332-9832-30a7d8f06975 admin admin] PATH_INFO: `/v3/projects/ab2dea97bdd048a291a4b4ee00b5de25` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:38.343968 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae7e01c-c349-4332-9832-30a7d8f06975 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab2dea97bdd048a291a4b4ee00b5de25)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:38.345963 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae7e01c-c349-4332-9832-30a7d8f06975 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:38.396871 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4ae7e01c-c349-4332-9832-30a7d8f06975 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab2dea97bdd048a291a4b4ee00b5de25', 'request_id': 'req-4ae7e01c-c349-4332-9832-30a7d8f06975'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:38.402952 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4ae7e01c-c349-4332-9832-30a7d8f06975 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab2dea97bdd048a291a4b4ee00b5de25', 'request_id': 'req-4ae7e01c-c349-4332-9832-30a7d8f06975'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:38.409654 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4ae7e01c-c349-4332-9832-30a7d8f06975 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab2dea97bdd048a291a4b4ee00b5de25', 'request_id': 'req-4ae7e01c-c349-4332-9832-30a7d8f06975'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:38.422367 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4ae7e01c-c349-4332-9832-30a7d8f06975 admin admin] The token cache is being invalidate because project ab2dea97bdd048a291a4b4ee00b5de25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:31:38.422652 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4ae7e01c-c349-4332-9832-30a7d8f06975 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:38.423306 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4ae7e01c-c349-4332-9832-30a7d8f06975 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:38.425434 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1350/2673] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:31:38 2022] DELETE /identity/v3/projects/ab2dea97bdd048a291a4b4ee00b5de25 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:38.429517 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2210f-a66b-4b62-8a58-aad5f5e31542 service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:38.439502 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2210f-a66b-4b62-8a58-aad5f5e31542 service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:38.457994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:38.500195 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c25dc9-eae8-4d58-80aa-89a38a383575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:38.500833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c25dc9-eae8-4d58-80aa-89a38a383575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:38.501100 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c25dc9-eae8-4d58-80aa-89a38a383575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:38.501368 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c25dc9-eae8-4d58-80aa-89a38a383575 admin admin] PATH_INFO: `/v3/users/2b3d828778d24de99bb6e4e299090972` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:38.502510 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c25dc9-eae8-4d58-80aa-89a38a383575 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b3d828778d24de99bb6e4e299090972)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:38.504451 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c25dc9-eae8-4d58-80aa-89a38a383575 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:38.550186 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-e5c25dc9-eae8-4d58-80aa-89a38a383575 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b3d828778d24de99bb6e4e299090972'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:38.558673 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-e5c25dc9-eae8-4d58-80aa-89a38a383575 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b3d828778d24de99bb6e4e299090972'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:38.559299 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-e5c25dc9-eae8-4d58-80aa-89a38a383575 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b3d828778d24de99bb6e4e299090972'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:38.571070 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-e5c25dc9-eae8-4d58-80aa-89a38a383575 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b3d828778d24de99bb6e4e299090972'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:38.577843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-e5c25dc9-eae8-4d58-80aa-89a38a383575 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b3d828778d24de99bb6e4e299090972'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:38.583657 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1324/2674] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:31:38 2022] DELETE /identity/v3/users/2b3d828778d24de99bb6e4e299090972 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:38.862973 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7e01c-c349-4332-9832-30a7d8f06975 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:38.864423 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4ae7e01c-c349-4332-9832-30a7d8f06975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:38.878877 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7e01c-c349-4332-9832-30a7d8f06975 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:38.937453 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee80a17-193e-4134-92fb-b97b96b4ad8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:38.938134 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee80a17-193e-4134-92fb-b97b96b4ad8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:38.938251 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee80a17-193e-4134-92fb-b97b96b4ad8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:38.938792 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee80a17-193e-4134-92fb-b97b96b4ad8e admin admin] PATH_INFO: `/v3/projects/888596d31b7d4725b4e2291872d4adf5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:38.939673 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee80a17-193e-4134-92fb-b97b96b4ad8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=888596d31b7d4725b4e2291872d4adf5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:38.941581 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee80a17-193e-4134-92fb-b97b96b4ad8e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:38.976343 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:38.994603 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4ee80a17-193e-4134-92fb-b97b96b4ad8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '888596d31b7d4725b4e2291872d4adf5', 'request_id': 'req-4ee80a17-193e-4134-92fb-b97b96b4ad8e'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:39.000080 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4ee80a17-193e-4134-92fb-b97b96b4ad8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '888596d31b7d4725b4e2291872d4adf5', 'request_id': 'req-4ee80a17-193e-4134-92fb-b97b96b4ad8e'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:39.006264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4ee80a17-193e-4134-92fb-b97b96b4ad8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '888596d31b7d4725b4e2291872d4adf5', 'request_id': 'req-4ee80a17-193e-4134-92fb-b97b96b4ad8e'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:39.020906 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4ee80a17-193e-4134-92fb-b97b96b4ad8e admin admin] The token cache is being invalidate because project 888596d31b7d4725b4e2291872d4adf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:31:39.021201 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4ee80a17-193e-4134-92fb-b97b96b4ad8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:39.021872 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4ee80a17-193e-4134-92fb-b97b96b4ad8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:39.023807 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1351/2675] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:31:38 2022] DELETE /identity/v3/projects/888596d31b7d4725b4e2291872d4adf5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:39.028791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10f2d6-3352-4dd4-9dba-be2f1e8f671b None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:39.029613 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10f2d6-3352-4dd4-9dba-be2f1e8f671b None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:39.029893 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10f2d6-3352-4dd4-9dba-be2f1e8f671b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:39.050785 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-ff10f2d6-3352-4dd4-9dba-be2f1e8f671b None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:39.052040 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-ff10f2d6-3352-4dd4-9dba-be2f1e8f671b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:39.110639 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1325/2676] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:31:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:39.118542 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75828802-08dd-4c37-8fed-407b77b4c9fe None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:39.118869 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75828802-08dd-4c37-8fed-407b77b4c9fe None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:39.119141 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75828802-08dd-4c37-8fed-407b77b4c9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:39.138570 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-75828802-08dd-4c37-8fed-407b77b4c9fe None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:39.139821 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-75828802-08dd-4c37-8fed-407b77b4c9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:39.148771 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1352/2677] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:31:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:39.153089 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff10f2d6-3352-4dd4-9dba-be2f1e8f671b None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:39.163264 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff10f2d6-3352-4dd4-9dba-be2f1e8f671b None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:39.167804 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24129222-a8a7-421d-abbe-136b631d3a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:39.168387 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24129222-a8a7-421d-abbe-136b631d3a22 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:39.168663 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24129222-a8a7-421d-abbe-136b631d3a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:39.168931 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24129222-a8a7-421d-abbe-136b631d3a22 admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:39.169564 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24129222-a8a7-421d-abbe-136b631d3a22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:31:39.169862 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24129222-a8a7-421d-abbe-136b631d3a22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:39.171425 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24129222-a8a7-421d-abbe-136b631d3a22 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:39.182740 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1326/2678] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:31:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:31:39.187020 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75828802-08dd-4c37-8fed-407b77b4c9fe None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:39.196926 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75828802-08dd-4c37-8fed-407b77b4c9fe None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:39.201564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981cd079-f007-4e8a-b738-258553fc4133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:39.202151 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981cd079-f007-4e8a-b738-258553fc4133 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:39.202430 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981cd079-f007-4e8a-b738-258553fc4133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:39.202703 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981cd079-f007-4e8a-b738-258553fc4133 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:39.203370 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981cd079-f007-4e8a-b738-258553fc4133 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:39.205304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981cd079-f007-4e8a-b738-258553fc4133 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:39.233126 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1353/2679] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:31:39 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:39.237457 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24129222-a8a7-421d-abbe-136b631d3a22 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:39.248190 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24129222-a8a7-421d-abbe-136b631d3a22 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:39.276156 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:39.308144 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c9ac4-230d-4fdd-ac95-ff4408ad5fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:39.308737 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c9ac4-230d-4fdd-ac95-ff4408ad5fee admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:39.309074 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c9ac4-230d-4fdd-ac95-ff4408ad5fee admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:39.309348 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c9ac4-230d-4fdd-ac95-ff4408ad5fee admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:39.309984 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c9ac4-230d-4fdd-ac95-ff4408ad5fee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:39.311908 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c9ac4-230d-4fdd-ac95-ff4408ad5fee admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:39.333713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1327/2680] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:31:39 2022] POST /identity/v3/users => generated 282 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:39.337917 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981cd079-f007-4e8a-b738-258553fc4133 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:39.348471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981cd079-f007-4e8a-b738-258553fc4133 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:39.353060 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92338cdc-a4bd-4eb3-a0c1-1fdf9d84ff59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:39.353659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92338cdc-a4bd-4eb3-a0c1-1fdf9d84ff59 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:39.353937 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92338cdc-a4bd-4eb3-a0c1-1fdf9d84ff59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:39.354210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92338cdc-a4bd-4eb3-a0c1-1fdf9d84ff59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:39.354840 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92338cdc-a4bd-4eb3-a0c1-1fdf9d84ff59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:39.356519 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92338cdc-a4bd-4eb3-a0c1-1fdf9d84ff59 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:39.366338 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1354/2681] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:39.370507 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c9ac4-230d-4fdd-ac95-ff4408ad5fee admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:39.380970 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c9ac4-230d-4fdd-ac95-ff4408ad5fee admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:39.385852 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf95a52-b111-4fbb-8fdd-b71f5f736bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:39.386438 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf95a52-b111-4fbb-8fdd-b71f5f736bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:39.386712 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf95a52-b111-4fbb-8fdd-b71f5f736bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:39.386986 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf95a52-b111-4fbb-8fdd-b71f5f736bb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:39.387650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf95a52-b111-4fbb-8fdd-b71f5f736bb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:39.389225 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf95a52-b111-4fbb-8fdd-b71f5f736bb1 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:39.400629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1328/2682] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:39.404903 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92338cdc-a4bd-4eb3-a0c1-1fdf9d84ff59 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:39.414982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92338cdc-a4bd-4eb3-a0c1-1fdf9d84ff59 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:39.419702 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fdc45-bc56-4c1f-9781-feade8ca6dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:39.420348 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fdc45-bc56-4c1f-9781-feade8ca6dfc admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:39.420631 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fdc45-bc56-4c1f-9781-feade8ca6dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:39.420906 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fdc45-bc56-4c1f-9781-feade8ca6dfc admin admin] PATH_INFO: `/v3/projects/985c3e67defb4b09a0babe660dfb8c29/users/906e31f3b6db4629abb843ee16eceda5/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:39.435281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1fdc45-bc56-4c1f-9781-feade8ca6dfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=985c3e67defb4b09a0babe660dfb8c29, user_id=906e31f3b6db4629abb843ee16eceda5, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:39.438018 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1fdc45-bc56-4c1f-9781-feade8ca6dfc admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:39.451490 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1355/2683] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:39 2022] PUT /identity/v3/projects/985c3e67defb4b09a0babe660dfb8c29/users/906e31f3b6db4629abb843ee16eceda5/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:39.462427 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544020e-cf57-466d-a79a-a17dae97ed30 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:39.462711 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544020e-cf57-466d-a79a-a17dae97ed30 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:39.463037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544020e-cf57-466d-a79a-a17dae97ed30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:39.490858 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-2544020e-cf57-466d-a79a-a17dae97ed30 None None] MFA Rules not processed for user `906e31f3b6db4629abb843ee16eceda5`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:39.492108 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-2544020e-cf57-466d-a79a-a17dae97ed30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:39.557098 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1329/2684] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:31:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:39.561591 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fdc45-bc56-4c1f-9781-feade8ca6dfc admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:39.621743 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fdc45-bc56-4c1f-9781-feade8ca6dfc admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:39.626391 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7abd8c7-bbeb-4fb3-809b-9cf014af02ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:39.626985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abd8c7-bbeb-4fb3-809b-9cf014af02ee admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:39.627269 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abd8c7-bbeb-4fb3-809b-9cf014af02ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:39.628137 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abd8c7-bbeb-4fb3-809b-9cf014af02ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:39.628770 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7abd8c7-bbeb-4fb3-809b-9cf014af02ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:39.630678 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7abd8c7-bbeb-4fb3-809b-9cf014af02ee admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:39.657924 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1356/2685] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:31:39 2022] POST /identity/v3/projects => generated 363 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:39.662039 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544020e-cf57-466d-a79a-a17dae97ed30 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:39.672520 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544020e-cf57-466d-a79a-a17dae97ed30 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:39.727877 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d15d3-bc92-42a8-a577-7a674c3215f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:39.728401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d15d3-bc92-42a8-a577-7a674c3215f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:39.728687 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d15d3-bc92-42a8-a577-7a674c3215f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:39.728933 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d15d3-bc92-42a8-a577-7a674c3215f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:39.729570 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d15d3-bc92-42a8-a577-7a674c3215f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:39.731469 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d15d3-bc92-42a8-a577-7a674c3215f7 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:39.753087 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1330/2686] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:31:39 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:39.757172 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7abd8c7-bbeb-4fb3-809b-9cf014af02ee admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:39.767373 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7abd8c7-bbeb-4fb3-809b-9cf014af02ee admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:39.772437 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3586f2-4ed1-46ab-915f-fd75eb914b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:39.773028 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3586f2-4ed1-46ab-915f-fd75eb914b5c admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:39.773303 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3586f2-4ed1-46ab-915f-fd75eb914b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:39.773827 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3586f2-4ed1-46ab-915f-fd75eb914b5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:39.774456 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3586f2-4ed1-46ab-915f-fd75eb914b5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:39.776081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3586f2-4ed1-46ab-915f-fd75eb914b5c admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:39.786742 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1357/2687] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:39.790962 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d15d3-bc92-42a8-a577-7a674c3215f7 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:39.801435 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d15d3-bc92-42a8-a577-7a674c3215f7 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:39.806025 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92760c2e-30b3-4640-be55-ee2eb2b093a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:39.806687 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92760c2e-30b3-4640-be55-ee2eb2b093a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:39.807204 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92760c2e-30b3-4640-be55-ee2eb2b093a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:39.807486 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92760c2e-30b3-4640-be55-ee2eb2b093a2 admin admin] PATH_INFO: `/v3/projects/4754dd5018b242f08e73e44d80ef549a/users/5c319d576744445b93bd8694c23391bf/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:39.824062 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92760c2e-30b3-4640-be55-ee2eb2b093a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4754dd5018b242f08e73e44d80ef549a, user_id=5c319d576744445b93bd8694c23391bf, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:39.826690 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92760c2e-30b3-4640-be55-ee2eb2b093a2 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:39.839297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1331/2688] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:39 2022] PUT /identity/v3/projects/4754dd5018b242f08e73e44d80ef549a/users/5c319d576744445b93bd8694c23391bf/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:39.843755 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3586f2-4ed1-46ab-915f-fd75eb914b5c admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:39.853801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3586f2-4ed1-46ab-915f-fd75eb914b5c admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:39.873637 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:39.912106 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bdfa66-7f0c-4bf6-97a9-09f4d864c24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:39.912708 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdfa66-7f0c-4bf6-97a9-09f4d864c24e admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:39.912987 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdfa66-7f0c-4bf6-97a9-09f4d864c24e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:39.913259 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdfa66-7f0c-4bf6-97a9-09f4d864c24e admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:39.913919 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bdfa66-7f0c-4bf6-97a9-09f4d864c24e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:39.915506 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bdfa66-7f0c-4bf6-97a9-09f4d864c24e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:39.925706 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1358/2689] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:39.929860 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92760c2e-30b3-4640-be55-ee2eb2b093a2 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:39.935244 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:39.941536 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92760c2e-30b3-4640-be55-ee2eb2b093a2 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:39.946677 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a06e2b-bd97-4a46-ab64-68774be1a844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:39.947343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a06e2b-bd97-4a46-ab64-68774be1a844 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:39.947652 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a06e2b-bd97-4a46-ab64-68774be1a844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:39.947923 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a06e2b-bd97-4a46-ab64-68774be1a844 admin admin] PATH_INFO: `/v3/projects/4754dd5018b242f08e73e44d80ef549a/users/5c319d576744445b93bd8694c23391bf/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:39.949836 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a06e2b-bd97-4a46-ab64-68774be1a844 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4754dd5018b242f08e73e44d80ef549a, user_id=5c319d576744445b93bd8694c23391bf, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:39.952573 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a06e2b-bd97-4a46-ab64-68774be1a844 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:39.962534 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1332/2690] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:39 2022] PUT /identity/v3/projects/4754dd5018b242f08e73e44d80ef549a/users/5c319d576744445b93bd8694c23391bf/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:39.974161 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0bf4a-34fb-40bb-98c7-01bb0d223dfc None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:39.974672 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0bf4a-34fb-40bb-98c7-01bb0d223dfc None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:39.974672 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0bf4a-34fb-40bb-98c7-01bb0d223dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:40.002781 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-06d0bf4a-34fb-40bb-98c7-01bb0d223dfc None None] MFA Rules not processed for user `5c319d576744445b93bd8694c23391bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:40.003963 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-06d0bf4a-34fb-40bb-98c7-01bb0d223dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:40.070723 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1359/2691] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:31:39 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:40.081927 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a06e2b-bd97-4a46-ab64-68774be1a844 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:40.083776 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-14a06e2b-bd97-4a46-ab64-68774be1a844 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:40.139680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a06e2b-bd97-4a46-ab64-68774be1a844 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:40.143223 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb8aa1-c22c-41e3-9e46-19f6c751210e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:40.143845 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb8aa1-c22c-41e3-9e46-19f6c751210e service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:40.144119 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb8aa1-c22c-41e3-9e46-19f6c751210e service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:40.144390 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb8aa1-c22c-41e3-9e46-19f6c751210e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:40.153296 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bb8aa1-c22c-41e3-9e46-19f6c751210e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:40.154747 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bb8aa1-c22c-41e3-9e46-19f6c751210e service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:40.164540 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1333/2692] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:40 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:31:40.574827 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0bf4a-34fb-40bb-98c7-01bb0d223dfc None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:40.635387 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0bf4a-34fb-40bb-98c7-01bb0d223dfc None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:40.639806 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b059ef-464a-46b9-a210-c41a1674ebe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:40.640420 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b059ef-464a-46b9-a210-c41a1674ebe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:40.640720 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b059ef-464a-46b9-a210-c41a1674ebe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:40.640969 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b059ef-464a-46b9-a210-c41a1674ebe8 admin admin] PATH_INFO: `/v3/users/906e31f3b6db4629abb843ee16eceda5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:40.642060 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b059ef-464a-46b9-a210-c41a1674ebe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=906e31f3b6db4629abb843ee16eceda5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:40.644489 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b059ef-464a-46b9-a210-c41a1674ebe8 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:40.692087 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e6b059ef-464a-46b9-a210-c41a1674ebe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '906e31f3b6db4629abb843ee16eceda5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:40.693256 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:40.700328 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e6b059ef-464a-46b9-a210-c41a1674ebe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '906e31f3b6db4629abb843ee16eceda5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:40.700992 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e6b059ef-464a-46b9-a210-c41a1674ebe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '906e31f3b6db4629abb843ee16eceda5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:40.701591 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e6b059ef-464a-46b9-a210-c41a1674ebe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '906e31f3b6db4629abb843ee16eceda5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:40.713747 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e6b059ef-464a-46b9-a210-c41a1674ebe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '906e31f3b6db4629abb843ee16eceda5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:40.726033 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1360/2693] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:31:40 2022] DELETE /identity/v3/users/906e31f3b6db4629abb843ee16eceda5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:40.730456 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba9c2e-80f7-4c67-b241-071e621a0cf2 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:40.730831 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba9c2e-80f7-4c67-b241-071e621a0cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:40.731221 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba9c2e-80f7-4c67-b241-071e621a0cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:40.750267 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-4eba9c2e-80f7-4c67-b241-071e621a0cf2 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:40.751467 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-4eba9c2e-80f7-4c67-b241-071e621a0cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:40.810534 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1334/2694] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:31:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:40.820731 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b059ef-464a-46b9-a210-c41a1674ebe8 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:40.822136 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e6b059ef-464a-46b9-a210-c41a1674ebe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:40.878235 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b059ef-464a-46b9-a210-c41a1674ebe8 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:40.881901 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd2d06-c696-4e80-bc05-c74622915552 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:40.882500 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd2d06-c696-4e80-bc05-c74622915552 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:40.882780 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd2d06-c696-4e80-bc05-c74622915552 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:40.883053 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd2d06-c696-4e80-bc05-c74622915552 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:40.892013 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd2d06-c696-4e80-bc05-c74622915552 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:40.894054 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd2d06-c696-4e80-bc05-c74622915552 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:40.903618 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1361/2695] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:31:41.181954 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba9c2e-80f7-4c67-b241-071e621a0cf2 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:41.183409 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-4eba9c2e-80f7-4c67-b241-071e621a0cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:41.197977 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba9c2e-80f7-4c67-b241-071e621a0cf2 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:41.202607 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430f86d-bcd1-4776-b2a6-757fca195eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:41.203235 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430f86d-bcd1-4776-b2a6-757fca195eb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:41.203512 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430f86d-bcd1-4776-b2a6-757fca195eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:41.204448 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430f86d-bcd1-4776-b2a6-757fca195eb9 admin admin] PATH_INFO: `/v3/projects/985c3e67defb4b09a0babe660dfb8c29` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:41.205908 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8430f86d-bcd1-4776-b2a6-757fca195eb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=985c3e67defb4b09a0babe660dfb8c29)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:41.207731 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8430f86d-bcd1-4776-b2a6-757fca195eb9 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:41.256395 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8430f86d-bcd1-4776-b2a6-757fca195eb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '985c3e67defb4b09a0babe660dfb8c29', 'request_id': 'req-8430f86d-bcd1-4776-b2a6-757fca195eb9'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:41.264572 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8430f86d-bcd1-4776-b2a6-757fca195eb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '985c3e67defb4b09a0babe660dfb8c29', 'request_id': 'req-8430f86d-bcd1-4776-b2a6-757fca195eb9'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:41.270343 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8430f86d-bcd1-4776-b2a6-757fca195eb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '985c3e67defb4b09a0babe660dfb8c29', 'request_id': 'req-8430f86d-bcd1-4776-b2a6-757fca195eb9'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:41.281854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8430f86d-bcd1-4776-b2a6-757fca195eb9 admin admin] The token cache is being invalidate because project 985c3e67defb4b09a0babe660dfb8c29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:31:41.282140 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8430f86d-bcd1-4776-b2a6-757fca195eb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:41.282769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8430f86d-bcd1-4776-b2a6-757fca195eb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:41.284274 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1335/2696] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:31:41 2022] DELETE /identity/v3/projects/985c3e67defb4b09a0babe660dfb8c29 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:41.288209 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd2d06-c696-4e80-bc05-c74622915552 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:41.297797 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd2d06-c696-4e80-bc05-c74622915552 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:41.354760 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101cfff-a49c-4f25-a912-8c40360277df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:41.355304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101cfff-a49c-4f25-a912-8c40360277df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:41.355685 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101cfff-a49c-4f25-a912-8c40360277df admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:41.355966 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101cfff-a49c-4f25-a912-8c40360277df admin admin] PATH_INFO: `/v3/users/5c319d576744445b93bd8694c23391bf` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:41.357383 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8101cfff-a49c-4f25-a912-8c40360277df admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c319d576744445b93bd8694c23391bf)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:41.359289 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8101cfff-a49c-4f25-a912-8c40360277df admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:41.406649 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8101cfff-a49c-4f25-a912-8c40360277df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c319d576744445b93bd8694c23391bf'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:41.414134 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8101cfff-a49c-4f25-a912-8c40360277df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c319d576744445b93bd8694c23391bf'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:41.414804 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8101cfff-a49c-4f25-a912-8c40360277df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c319d576744445b93bd8694c23391bf'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:41.415413 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8101cfff-a49c-4f25-a912-8c40360277df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c319d576744445b93bd8694c23391bf'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:41.426101 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:41.428547 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8101cfff-a49c-4f25-a912-8c40360277df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c319d576744445b93bd8694c23391bf'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:41.441876 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1362/2697] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:31:41 2022] DELETE /identity/v3/users/5c319d576744445b93bd8694c23391bf => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:41.715129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430f86d-bcd1-4776-b2a6-757fca195eb9 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:41.717476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-8430f86d-bcd1-4776-b2a6-757fca195eb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:41.731011 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430f86d-bcd1-4776-b2a6-757fca195eb9 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:41.744349 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:41.792659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b4fb5-d1e5-469d-94c8-aebad1ebcaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:41.793298 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b4fb5-d1e5-469d-94c8-aebad1ebcaa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:41.793574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b4fb5-d1e5-469d-94c8-aebad1ebcaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:41.793846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b4fb5-d1e5-469d-94c8-aebad1ebcaa0 admin admin] PATH_INFO: `/v3/projects/4754dd5018b242f08e73e44d80ef549a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:41.794959 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b4fb5-d1e5-469d-94c8-aebad1ebcaa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4754dd5018b242f08e73e44d80ef549a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:41.796892 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b4fb5-d1e5-469d-94c8-aebad1ebcaa0 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:41.842586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-965b4fb5-d1e5-469d-94c8-aebad1ebcaa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4754dd5018b242f08e73e44d80ef549a', 'request_id': 'req-965b4fb5-d1e5-469d-94c8-aebad1ebcaa0'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:41.851584 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-965b4fb5-d1e5-469d-94c8-aebad1ebcaa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4754dd5018b242f08e73e44d80ef549a', 'request_id': 'req-965b4fb5-d1e5-469d-94c8-aebad1ebcaa0'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:41.857367 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-965b4fb5-d1e5-469d-94c8-aebad1ebcaa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4754dd5018b242f08e73e44d80ef549a', 'request_id': 'req-965b4fb5-d1e5-469d-94c8-aebad1ebcaa0'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:41.868494 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-965b4fb5-d1e5-469d-94c8-aebad1ebcaa0 admin admin] The token cache is being invalidate because project 4754dd5018b242f08e73e44d80ef549a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:31:41.868775 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-965b4fb5-d1e5-469d-94c8-aebad1ebcaa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:41.869403 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-965b4fb5-d1e5-469d-94c8-aebad1ebcaa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:41.870849 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1336/2698] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:31:41 2022] DELETE /identity/v3/projects/4754dd5018b242f08e73e44d80ef549a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:41.875895 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cec8f-dee9-4029-9c06-5d23b6474973 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:41.876181 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cec8f-dee9-4029-9c06-5d23b6474973 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:41.876455 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cec8f-dee9-4029-9c06-5d23b6474973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:41.896159 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-b16cec8f-dee9-4029-9c06-5d23b6474973 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:41.897274 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b16cec8f-dee9-4029-9c06-5d23b6474973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:41.954101 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1363/2699] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:41.962623 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac62fcf-1af9-4eef-baac-5a04954f4bfc None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:41.962907 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac62fcf-1af9-4eef-baac-5a04954f4bfc None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:41.963181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac62fcf-1af9-4eef-baac-5a04954f4bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:41.983293 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-fac62fcf-1af9-4eef-baac-5a04954f4bfc None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:41.984540 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-fac62fcf-1af9-4eef-baac-5a04954f4bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:41.992518 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1337/2700] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:41.996675 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16cec8f-dee9-4029-9c06-5d23b6474973 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:42.006275 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16cec8f-dee9-4029-9c06-5d23b6474973 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:42.010803 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096ecec-bf27-4ffe-b469-74e789c7923a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:42.011392 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096ecec-bf27-4ffe-b469-74e789c7923a admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:42.011713 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096ecec-bf27-4ffe-b469-74e789c7923a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:42.011986 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096ecec-bf27-4ffe-b469-74e789c7923a admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:42.012625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2096ecec-bf27-4ffe-b469-74e789c7923a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:31:42.012928 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2096ecec-bf27-4ffe-b469-74e789c7923a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:42.014509 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2096ecec-bf27-4ffe-b469-74e789c7923a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:42.026679 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1364/2701] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:31:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:31:42.031146 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac62fcf-1af9-4eef-baac-5a04954f4bfc None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:42.041334 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac62fcf-1af9-4eef-baac-5a04954f4bfc None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:42.046260 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c84e0d-29fa-416a-96fc-218e11a66da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:42.046847 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c84e0d-29fa-416a-96fc-218e11a66da2 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:42.047128 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c84e0d-29fa-416a-96fc-218e11a66da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:42.047399 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c84e0d-29fa-416a-96fc-218e11a66da2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:42.048068 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c84e0d-29fa-416a-96fc-218e11a66da2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:42.049954 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c84e0d-29fa-416a-96fc-218e11a66da2 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:42.085809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1338/2702] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:31:42 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:42.090089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096ecec-bf27-4ffe-b469-74e789c7923a admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:42.100727 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096ecec-bf27-4ffe-b469-74e789c7923a admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:42.155845 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f357bf42-9983-4e1a-b8e5-3dbbb01c6753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:42.156378 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357bf42-9983-4e1a-b8e5-3dbbb01c6753 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:42.156652 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357bf42-9983-4e1a-b8e5-3dbbb01c6753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:42.156920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357bf42-9983-4e1a-b8e5-3dbbb01c6753 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:42.157541 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f357bf42-9983-4e1a-b8e5-3dbbb01c6753 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:42.160115 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f357bf42-9983-4e1a-b8e5-3dbbb01c6753 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:42.180935 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1365/2703] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:31:42 2022] POST /identity/v3/users => generated 283 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:42.185028 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c84e0d-29fa-416a-96fc-218e11a66da2 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:42.196217 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c84e0d-29fa-416a-96fc-218e11a66da2 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:42.200864 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284256b-d965-4b37-9e0f-0a9b526dc5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:42.201483 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284256b-d965-4b37-9e0f-0a9b526dc5a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:42.201761 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284256b-d965-4b37-9e0f-0a9b526dc5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:42.202033 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284256b-d965-4b37-9e0f-0a9b526dc5a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:42.202666 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d284256b-d965-4b37-9e0f-0a9b526dc5a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:42.204288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d284256b-d965-4b37-9e0f-0a9b526dc5a3 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:42.214595 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1339/2704] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:42.218479 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f357bf42-9983-4e1a-b8e5-3dbbb01c6753 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:42.228818 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f357bf42-9983-4e1a-b8e5-3dbbb01c6753 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:42.233311 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0002d-f1f6-4b79-bd02-6a69d2f9189a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:42.233892 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0002d-f1f6-4b79-bd02-6a69d2f9189a admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:42.234169 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0002d-f1f6-4b79-bd02-6a69d2f9189a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:42.234437 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0002d-f1f6-4b79-bd02-6a69d2f9189a admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:42.235062 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a0002d-f1f6-4b79-bd02-6a69d2f9189a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:42.236702 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a0002d-f1f6-4b79-bd02-6a69d2f9189a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:42.249407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1366/2705] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:31:42.251731 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284256b-d965-4b37-9e0f-0a9b526dc5a3 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:42.262093 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284256b-d965-4b37-9e0f-0a9b526dc5a3 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:42.266583 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48358f-b608-402b-88ec-167e69b2b32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:42.267221 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48358f-b608-402b-88ec-167e69b2b32b admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:42.267498 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48358f-b608-402b-88ec-167e69b2b32b admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:42.267801 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48358f-b608-402b-88ec-167e69b2b32b admin admin] PATH_INFO: `/v3/projects/a0ddeca7668e4e10aa43d14b63cf6473/users/55cba7affb0a4e659284f954257b4b34/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:42.282826 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b48358f-b608-402b-88ec-167e69b2b32b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0ddeca7668e4e10aa43d14b63cf6473, user_id=55cba7affb0a4e659284f954257b4b34, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:42.285598 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b48358f-b608-402b-88ec-167e69b2b32b admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:42.298119 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1340/2706] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:42 2022] PUT /identity/v3/projects/a0ddeca7668e4e10aa43d14b63cf6473/users/55cba7affb0a4e659284f954257b4b34/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:42.309880 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533c698-70e6-458b-b85f-382e41702801 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:42.310450 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533c698-70e6-458b-b85f-382e41702801 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:42.310612 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533c698-70e6-458b-b85f-382e41702801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:42.338821 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-5533c698-70e6-458b-b85f-382e41702801 None None] MFA Rules not processed for user `55cba7affb0a4e659284f954257b4b34`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:42.340088 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5533c698-70e6-458b-b85f-382e41702801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:42.382733 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:42.408543 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1367/2707] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:31:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:42.412846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48358f-b608-402b-88ec-167e69b2b32b admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:42.457472 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:42.476583 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48358f-b608-402b-88ec-167e69b2b32b admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:42.481389 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572640fe-d23f-47a8-ac7a-4fd97af2ba76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:42.481984 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572640fe-d23f-47a8-ac7a-4fd97af2ba76 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:42.482263 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572640fe-d23f-47a8-ac7a-4fd97af2ba76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:42.482537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572640fe-d23f-47a8-ac7a-4fd97af2ba76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:42.483182 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572640fe-d23f-47a8-ac7a-4fd97af2ba76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:42.485114 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572640fe-d23f-47a8-ac7a-4fd97af2ba76 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:42.514040 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1341/2708] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:31:42 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:42.518844 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533c698-70e6-458b-b85f-382e41702801 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:42.530165 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533c698-70e6-458b-b85f-382e41702801 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:42.586554 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714b873-ad86-416c-836f-5b125618e983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:42.587216 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714b873-ad86-416c-836f-5b125618e983 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:42.587500 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714b873-ad86-416c-836f-5b125618e983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:42.587827 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714b873-ad86-416c-836f-5b125618e983 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:42.588462 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714b873-ad86-416c-836f-5b125618e983 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:42.590388 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714b873-ad86-416c-836f-5b125618e983 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:42.612428 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1368/2709] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:31:42 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:42.616488 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572640fe-d23f-47a8-ac7a-4fd97af2ba76 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:42.627120 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572640fe-d23f-47a8-ac7a-4fd97af2ba76 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:42.632419 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b1cdf-2174-4256-9772-8faad5af47fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:42.633020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b1cdf-2174-4256-9772-8faad5af47fb admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:42.633300 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b1cdf-2174-4256-9772-8faad5af47fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:42.633698 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b1cdf-2174-4256-9772-8faad5af47fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:42.634349 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b1cdf-2174-4256-9772-8faad5af47fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:42.635982 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b1cdf-2174-4256-9772-8faad5af47fb admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:42.647628 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1342/2710] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:42.652010 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714b873-ad86-416c-836f-5b125618e983 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:42.664145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714b873-ad86-416c-836f-5b125618e983 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:42.668931 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80cb91-5931-43a0-aa3d-abb200148dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:42.669573 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80cb91-5931-43a0-aa3d-abb200148dd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:42.669855 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80cb91-5931-43a0-aa3d-abb200148dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:42.670131 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80cb91-5931-43a0-aa3d-abb200148dd0 admin admin] PATH_INFO: `/v3/projects/48ed4ed05a89415baf8a7d604c49d70e/users/9bad50a1cab84e8095badad32c98b2dc/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:42.691408 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f80cb91-5931-43a0-aa3d-abb200148dd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48ed4ed05a89415baf8a7d604c49d70e, user_id=9bad50a1cab84e8095badad32c98b2dc, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:42.694091 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f80cb91-5931-43a0-aa3d-abb200148dd0 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:42.706702 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1369/2711] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:42 2022] PUT /identity/v3/projects/48ed4ed05a89415baf8a7d604c49d70e/users/9bad50a1cab84e8095badad32c98b2dc/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:42.710662 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b1cdf-2174-4256-9772-8faad5af47fb admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:42.720589 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b1cdf-2174-4256-9772-8faad5af47fb admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:42.778280 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b0a68-8b85-4724-b086-57c886a2f41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:42.779739 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b0a68-8b85-4724-b086-57c886a2f41d admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:42.779739 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b0a68-8b85-4724-b086-57c886a2f41d admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:42.779739 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b0a68-8b85-4724-b086-57c886a2f41d admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:42.780176 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511b0a68-8b85-4724-b086-57c886a2f41d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:42.781746 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511b0a68-8b85-4724-b086-57c886a2f41d admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:42.791736 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1343/2712] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:42.795887 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80cb91-5931-43a0-aa3d-abb200148dd0 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:42.806856 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80cb91-5931-43a0-aa3d-abb200148dd0 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:42.811643 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0359fd-185c-4cc1-8f99-36d6216bf04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:42.812305 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0359fd-185c-4cc1-8f99-36d6216bf04c admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:42.812586 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0359fd-185c-4cc1-8f99-36d6216bf04c admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:42.812863 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0359fd-185c-4cc1-8f99-36d6216bf04c admin admin] PATH_INFO: `/v3/projects/48ed4ed05a89415baf8a7d604c49d70e/users/9bad50a1cab84e8095badad32c98b2dc/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:42.815403 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0359fd-185c-4cc1-8f99-36d6216bf04c admin admin] RBAC: Authorizing `identity:create_grant(project_id=48ed4ed05a89415baf8a7d604c49d70e, user_id=9bad50a1cab84e8095badad32c98b2dc, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:42.818125 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0359fd-185c-4cc1-8f99-36d6216bf04c admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:42.828492 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1370/2713] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:42 2022] PUT /identity/v3/projects/48ed4ed05a89415baf8a7d604c49d70e/users/9bad50a1cab84e8095badad32c98b2dc/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:42.839638 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81093ba4-0da7-47b1-baf4-12f9af6fd8c1 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:42.839923 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81093ba4-0da7-47b1-baf4-12f9af6fd8c1 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:42.840196 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81093ba4-0da7-47b1-baf4-12f9af6fd8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:42.869037 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-81093ba4-0da7-47b1-baf4-12f9af6fd8c1 None None] MFA Rules not processed for user `9bad50a1cab84e8095badad32c98b2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:42.870254 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-81093ba4-0da7-47b1-baf4-12f9af6fd8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:42.940537 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1344/2714] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:31:42 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:42.953255 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0359fd-185c-4cc1-8f99-36d6216bf04c admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:42.955089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4f0359fd-185c-4cc1-8f99-36d6216bf04c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:43.013325 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0359fd-185c-4cc1-8f99-36d6216bf04c admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:43.017039 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f24da-0659-4c46-8fe5-2265c9cdd2a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:43.017644 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f24da-0659-4c46-8fe5-2265c9cdd2a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:43.018039 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f24da-0659-4c46-8fe5-2265c9cdd2a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:43.018317 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f24da-0659-4c46-8fe5-2265c9cdd2a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:43.027641 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280f24da-0659-4c46-8fe5-2265c9cdd2a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:43.029070 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280f24da-0659-4c46-8fe5-2265c9cdd2a7 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:43.038847 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1371/2715] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:42 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:31:43.114423 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81093ba4-0da7-47b1-baf4-12f9af6fd8c1 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:43.124132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81093ba4-0da7-47b1-baf4-12f9af6fd8c1 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:43.127917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426fc5b-f99f-434b-b492-a2a63ee92a72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:43.128505 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426fc5b-f99f-434b-b492-a2a63ee92a72 service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:43.128782 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426fc5b-f99f-434b-b492-a2a63ee92a72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:43.129052 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426fc5b-f99f-434b-b492-a2a63ee92a72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:43.162311 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:43.185912 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7426fc5b-f99f-434b-b492-a2a63ee92a72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:43.187380 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7426fc5b-f99f-434b-b492-a2a63ee92a72 service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:43.196374 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1345/2716] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:31:46.095410 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f24da-0659-4c46-8fe5-2265c9cdd2a7 service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:46.135075 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:46.157835 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f24da-0659-4c46-8fe5-2265c9cdd2a7 service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:46.162425 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74860b75-0eeb-41f1-8307-fa45483efb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:46.163045 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74860b75-0eeb-41f1-8307-fa45483efb55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:46.163328 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74860b75-0eeb-41f1-8307-fa45483efb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:46.163644 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74860b75-0eeb-41f1-8307-fa45483efb55 admin admin] PATH_INFO: `/v3/users/55cba7affb0a4e659284f954257b4b34` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:46.164761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74860b75-0eeb-41f1-8307-fa45483efb55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55cba7affb0a4e659284f954257b4b34)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:46.166673 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74860b75-0eeb-41f1-8307-fa45483efb55 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:46.211426 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-74860b75-0eeb-41f1-8307-fa45483efb55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55cba7affb0a4e659284f954257b4b34'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:46.218156 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-74860b75-0eeb-41f1-8307-fa45483efb55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55cba7affb0a4e659284f954257b4b34'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:46.218803 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-74860b75-0eeb-41f1-8307-fa45483efb55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55cba7affb0a4e659284f954257b4b34'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:46.219418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-74860b75-0eeb-41f1-8307-fa45483efb55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55cba7affb0a4e659284f954257b4b34'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:46.231615 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-74860b75-0eeb-41f1-8307-fa45483efb55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55cba7affb0a4e659284f954257b4b34'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:46.244209 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1372/2717] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:31:46 2022] DELETE /identity/v3/users/55cba7affb0a4e659284f954257b4b34 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:31:46.249117 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4ace4-e17a-45c6-a69a-31d1d75994ef None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:46.249675 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4ace4-e17a-45c6-a69a-31d1d75994ef None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:46.249675 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4ace4-e17a-45c6-a69a-31d1d75994ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:46.269591 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-e2d4ace4-e17a-45c6-a69a-31d1d75994ef None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:46.270772 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-e2d4ace4-e17a-45c6-a69a-31d1d75994ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:46.328974 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1346/2718] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:46.339192 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74860b75-0eeb-41f1-8307-fa45483efb55 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:46.340713 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-74860b75-0eeb-41f1-8307-fa45483efb55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:46.398084 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74860b75-0eeb-41f1-8307-fa45483efb55 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:46.402020 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57208cd-04a3-4446-b8d5-9a0bb764b17a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:46.402629 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57208cd-04a3-4446-b8d5-9a0bb764b17a service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:46.402907 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57208cd-04a3-4446-b8d5-9a0bb764b17a service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:46.403211 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57208cd-04a3-4446-b8d5-9a0bb764b17a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:46.412459 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57208cd-04a3-4446-b8d5-9a0bb764b17a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:46.413913 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57208cd-04a3-4446-b8d5-9a0bb764b17a service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:46.423104 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1373/2719] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:31:46.697500 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4ace4-e17a-45c6-a69a-31d1d75994ef None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:46.698981 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-e2d4ace4-e17a-45c6-a69a-31d1d75994ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:46.712317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4ace4-e17a-45c6-a69a-31d1d75994ef None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:46.716994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1ae08-c229-4470-ac6d-50716c68b97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:46.717650 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1ae08-c229-4470-ac6d-50716c68b97f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:46.717929 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1ae08-c229-4470-ac6d-50716c68b97f admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:46.718231 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1ae08-c229-4470-ac6d-50716c68b97f admin admin] PATH_INFO: `/v3/projects/a0ddeca7668e4e10aa43d14b63cf6473` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:46.719369 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae1ae08-c229-4470-ac6d-50716c68b97f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0ddeca7668e4e10aa43d14b63cf6473)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:46.721920 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae1ae08-c229-4470-ac6d-50716c68b97f admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:46.769474 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8ae1ae08-c229-4470-ac6d-50716c68b97f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0ddeca7668e4e10aa43d14b63cf6473', 'request_id': 'req-8ae1ae08-c229-4470-ac6d-50716c68b97f'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:46.777807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8ae1ae08-c229-4470-ac6d-50716c68b97f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0ddeca7668e4e10aa43d14b63cf6473', 'request_id': 'req-8ae1ae08-c229-4470-ac6d-50716c68b97f'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:46.784005 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8ae1ae08-c229-4470-ac6d-50716c68b97f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0ddeca7668e4e10aa43d14b63cf6473', 'request_id': 'req-8ae1ae08-c229-4470-ac6d-50716c68b97f'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:46.794978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8ae1ae08-c229-4470-ac6d-50716c68b97f admin admin] The token cache is being invalidate because project a0ddeca7668e4e10aa43d14b63cf6473 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:31:46.795242 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8ae1ae08-c229-4470-ac6d-50716c68b97f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:46.795955 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8ae1ae08-c229-4470-ac6d-50716c68b97f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:46.797425 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1347/2720] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:31:46 2022] DELETE /identity/v3/projects/a0ddeca7668e4e10aa43d14b63cf6473 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:31:46.801969 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57208cd-04a3-4446-b8d5-9a0bb764b17a service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:46.811580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57208cd-04a3-4446-b8d5-9a0bb764b17a service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:46.866928 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7974c8-d382-4887-8864-eec5a86999f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:46.867590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7974c8-d382-4887-8864-eec5a86999f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:46.867868 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7974c8-d382-4887-8864-eec5a86999f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:46.868137 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7974c8-d382-4887-8864-eec5a86999f7 admin admin] PATH_INFO: `/v3/users/9bad50a1cab84e8095badad32c98b2dc` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:46.869238 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7974c8-d382-4887-8864-eec5a86999f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bad50a1cab84e8095badad32c98b2dc)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:46.871145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7974c8-d382-4887-8864-eec5a86999f7 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:46.872636 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:46.922917 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ca7974c8-d382-4887-8864-eec5a86999f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bad50a1cab84e8095badad32c98b2dc'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:46.930244 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ca7974c8-d382-4887-8864-eec5a86999f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bad50a1cab84e8095badad32c98b2dc'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:46.930896 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ca7974c8-d382-4887-8864-eec5a86999f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bad50a1cab84e8095badad32c98b2dc'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:46.931491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ca7974c8-d382-4887-8864-eec5a86999f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bad50a1cab84e8095badad32c98b2dc'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:46.942981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ca7974c8-d382-4887-8864-eec5a86999f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bad50a1cab84e8095badad32c98b2dc'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:46.956253 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1374/2721] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:31:46 2022] DELETE /identity/v3/users/9bad50a1cab84e8095badad32c98b2dc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:47.229745 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1ae08-c229-4470-ac6d-50716c68b97f admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:47.231187 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-8ae1ae08-c229-4470-ac6d-50716c68b97f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:47.244433 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1ae08-c229-4470-ac6d-50716c68b97f admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:47.308723 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78392c33-4c86-44a5-a879-4228766c9636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:47.309611 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78392c33-4c86-44a5-a879-4228766c9636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:47.309728 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78392c33-4c86-44a5-a879-4228766c9636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:47.310019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78392c33-4c86-44a5-a879-4228766c9636 admin admin] PATH_INFO: `/v3/projects/48ed4ed05a89415baf8a7d604c49d70e` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:47.311238 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78392c33-4c86-44a5-a879-4228766c9636 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48ed4ed05a89415baf8a7d604c49d70e)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:47.313286 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78392c33-4c86-44a5-a879-4228766c9636 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:47.314801 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:47.362767 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-78392c33-4c86-44a5-a879-4228766c9636 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48ed4ed05a89415baf8a7d604c49d70e', 'request_id': 'req-78392c33-4c86-44a5-a879-4228766c9636'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:47.370577 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-78392c33-4c86-44a5-a879-4228766c9636 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48ed4ed05a89415baf8a7d604c49d70e', 'request_id': 'req-78392c33-4c86-44a5-a879-4228766c9636'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:47.376593 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-78392c33-4c86-44a5-a879-4228766c9636 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48ed4ed05a89415baf8a7d604c49d70e', 'request_id': 'req-78392c33-4c86-44a5-a879-4228766c9636'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:47.387874 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-78392c33-4c86-44a5-a879-4228766c9636 admin admin] The token cache is being invalidate because project 48ed4ed05a89415baf8a7d604c49d70e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:31:47.388159 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-78392c33-4c86-44a5-a879-4228766c9636 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:47.388805 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-78392c33-4c86-44a5-a879-4228766c9636 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:47.390294 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1348/2722] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:31:47 2022] DELETE /identity/v3/projects/48ed4ed05a89415baf8a7d604c49d70e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:47.395793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f90ad9-53df-4eb3-a4e9-f815372813bd None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:47.396264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f90ad9-53df-4eb3-a4e9-f815372813bd None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:47.396378 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f90ad9-53df-4eb3-a4e9-f815372813bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:47.416629 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-c6f90ad9-53df-4eb3-a4e9-f815372813bd None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:47.417831 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c6f90ad9-53df-4eb3-a4e9-f815372813bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:47.476289 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1375/2723] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:31:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:47.484102 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd408a9-f067-4158-ae0c-e73f04bb6b6b None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:47.484391 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd408a9-f067-4158-ae0c-e73f04bb6b6b None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:47.484670 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd408a9-f067-4158-ae0c-e73f04bb6b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:47.505093 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-1cd408a9-f067-4158-ae0c-e73f04bb6b6b None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:47.506307 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-1cd408a9-f067-4158-ae0c-e73f04bb6b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:47.514295 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1349/2724] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:31:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:47.519391 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f90ad9-53df-4eb3-a4e9-f815372813bd None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:47.529388 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f90ad9-53df-4eb3-a4e9-f815372813bd None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:47.534035 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53436116-cce1-4937-89f5-7a9d32262384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:47.534624 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53436116-cce1-4937-89f5-7a9d32262384 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:47.534908 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53436116-cce1-4937-89f5-7a9d32262384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:47.535210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53436116-cce1-4937-89f5-7a9d32262384 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:47.535876 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53436116-cce1-4937-89f5-7a9d32262384 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:31:47.536183 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53436116-cce1-4937-89f5-7a9d32262384 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:47.537774 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53436116-cce1-4937-89f5-7a9d32262384 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:47.549302 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1376/2725] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:31:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:31:47.553468 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd408a9-f067-4158-ae0c-e73f04bb6b6b None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:47.562764 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd408a9-f067-4158-ae0c-e73f04bb6b6b None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:47.567899 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aafbc6-eb57-46f9-a57a-007ed52df9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:47.568481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aafbc6-eb57-46f9-a57a-007ed52df9e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:47.568763 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aafbc6-eb57-46f9-a57a-007ed52df9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:47.569039 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aafbc6-eb57-46f9-a57a-007ed52df9e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:47.569681 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aafbc6-eb57-46f9-a57a-007ed52df9e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:47.571603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aafbc6-eb57-46f9-a57a-007ed52df9e3 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:47.600045 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1350/2726] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:31:47 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:47.604149 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53436116-cce1-4937-89f5-7a9d32262384 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:47.613927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53436116-cce1-4937-89f5-7a9d32262384 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:47.669822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6d6bc-ab24-4af6-aaeb-6f7d1c067c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:47.670423 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6d6bc-ab24-4af6-aaeb-6f7d1c067c00 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:47.670699 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6d6bc-ab24-4af6-aaeb-6f7d1c067c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:47.670970 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6d6bc-ab24-4af6-aaeb-6f7d1c067c00 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:47.671616 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6d6bc-ab24-4af6-aaeb-6f7d1c067c00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:47.673508 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6d6bc-ab24-4af6-aaeb-6f7d1c067c00 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:47.684242 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:47.696877 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1377/2727] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:31:47 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:47.701192 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aafbc6-eb57-46f9-a57a-007ed52df9e3 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:47.712023 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aafbc6-eb57-46f9-a57a-007ed52df9e3 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:47.716645 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eadc2b-99d4-4068-bfcb-df867f8c074e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:47.717241 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eadc2b-99d4-4068-bfcb-df867f8c074e admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:47.717520 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eadc2b-99d4-4068-bfcb-df867f8c074e admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:47.717795 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eadc2b-99d4-4068-bfcb-df867f8c074e admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:47.718432 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eadc2b-99d4-4068-bfcb-df867f8c074e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:47.720047 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eadc2b-99d4-4068-bfcb-df867f8c074e admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:47.729881 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1351/2728] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:47.733686 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6d6bc-ab24-4af6-aaeb-6f7d1c067c00 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:47.743706 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6d6bc-ab24-4af6-aaeb-6f7d1c067c00 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:47.748377 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4ab85-b9c3-4001-ae00-ff6bdf551ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:47.748988 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4ab85-b9c3-4001-ae00-ff6bdf551ae6 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:47.749250 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4ab85-b9c3-4001-ae00-ff6bdf551ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:47.749570 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4ab85-b9c3-4001-ae00-ff6bdf551ae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:47.750455 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed4ab85-b9c3-4001-ae00-ff6bdf551ae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:47.752664 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed4ab85-b9c3-4001-ae00-ff6bdf551ae6 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:47.763405 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1378/2729] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:47.767418 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eadc2b-99d4-4068-bfcb-df867f8c074e admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:47.778774 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eadc2b-99d4-4068-bfcb-df867f8c074e admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:47.783615 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26c9b9-e3a9-4f2c-9afe-f029c382b8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:47.784321 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26c9b9-e3a9-4f2c-9afe-f029c382b8da admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:47.784600 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26c9b9-e3a9-4f2c-9afe-f029c382b8da admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:47.784876 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26c9b9-e3a9-4f2c-9afe-f029c382b8da admin admin] PATH_INFO: `/v3/projects/651ce115cc4e4ea98017ec042e650681/users/eca0f5fd566b48b8b3b6b21b09306112/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:47.800177 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e26c9b9-e3a9-4f2c-9afe-f029c382b8da admin admin] RBAC: Authorizing `identity:create_grant(project_id=651ce115cc4e4ea98017ec042e650681, user_id=eca0f5fd566b48b8b3b6b21b09306112, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:47.802972 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e26c9b9-e3a9-4f2c-9afe-f029c382b8da admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:47.817522 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1352/2730] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:47 2022] PUT /identity/v3/projects/651ce115cc4e4ea98017ec042e650681/users/eca0f5fd566b48b8b3b6b21b09306112/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:47.829071 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a50c1-a1b4-425b-831f-6019bde94afc None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:47.829361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a50c1-a1b4-425b-831f-6019bde94afc None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:47.829637 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a50c1-a1b4-425b-831f-6019bde94afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:47.857670 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-f34a50c1-a1b4-425b-831f-6019bde94afc None None] MFA Rules not processed for user `eca0f5fd566b48b8b3b6b21b09306112`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:47.858924 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f34a50c1-a1b4-425b-831f-6019bde94afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:47.923178 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1379/2731] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:31:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:47.928297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26c9b9-e3a9-4f2c-9afe-f029c382b8da admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:47.988442 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26c9b9-e3a9-4f2c-9afe-f029c382b8da admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:47.993197 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef0a24-e684-49d9-a1bb-19d843e4db6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:47.993830 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef0a24-e684-49d9-a1bb-19d843e4db6b admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:47.994114 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef0a24-e684-49d9-a1bb-19d843e4db6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:47.994390 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef0a24-e684-49d9-a1bb-19d843e4db6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:47.995027 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef0a24-e684-49d9-a1bb-19d843e4db6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:47.996955 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef0a24-e684-49d9-a1bb-19d843e4db6b admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:48.025178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1353/2732] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:31:47 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:48.029617 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a50c1-a1b4-425b-831f-6019bde94afc None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:48.040399 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a50c1-a1b4-425b-831f-6019bde94afc None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:48.097222 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2d7d8-2267-43b8-a3d6-2222b74eb4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:48.097746 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2d7d8-2267-43b8-a3d6-2222b74eb4af admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:48.098027 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2d7d8-2267-43b8-a3d6-2222b74eb4af admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:48.098300 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2d7d8-2267-43b8-a3d6-2222b74eb4af admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:48.098970 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2d7d8-2267-43b8-a3d6-2222b74eb4af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:48.100911 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2d7d8-2267-43b8-a3d6-2222b74eb4af admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:48.122622 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1380/2733] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:31:48 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:48.126724 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef0a24-e684-49d9-a1bb-19d843e4db6b admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:48.137964 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef0a24-e684-49d9-a1bb-19d843e4db6b admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:48.142748 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9041c1-8785-45e5-ab05-7722b186b464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:48.143353 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9041c1-8785-45e5-ab05-7722b186b464 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:48.143679 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9041c1-8785-45e5-ab05-7722b186b464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:48.143954 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9041c1-8785-45e5-ab05-7722b186b464 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:48.144599 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9041c1-8785-45e5-ab05-7722b186b464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:48.146174 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9041c1-8785-45e5-ab05-7722b186b464 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:48.151842 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:48.162498 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1354/2734] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:48.163673 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2d7d8-2267-43b8-a3d6-2222b74eb4af admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:48.173635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2d7d8-2267-43b8-a3d6-2222b74eb4af admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:48.178821 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427eb561-53ab-48b4-b979-c4da45b2a3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:48.179441 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427eb561-53ab-48b4-b979-c4da45b2a3b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:48.179755 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427eb561-53ab-48b4-b979-c4da45b2a3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:48.180031 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427eb561-53ab-48b4-b979-c4da45b2a3b9 admin admin] PATH_INFO: `/v3/projects/e69ce85635654b9e98a22a8cab380464/users/beba6736cec24c80b426977c7cbd23d7/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:48.194300 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427eb561-53ab-48b4-b979-c4da45b2a3b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e69ce85635654b9e98a22a8cab380464, user_id=beba6736cec24c80b426977c7cbd23d7, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:48.196990 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427eb561-53ab-48b4-b979-c4da45b2a3b9 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:48.209336 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1381/2735] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:48 2022] PUT /identity/v3/projects/e69ce85635654b9e98a22a8cab380464/users/beba6736cec24c80b426977c7cbd23d7/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:48.213364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9041c1-8785-45e5-ab05-7722b186b464 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:48.224300 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9041c1-8785-45e5-ab05-7722b186b464 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:48.280174 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5516d8e5-e97c-4d5b-a9a6-5365a1e878d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:48.280776 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5516d8e5-e97c-4d5b-a9a6-5365a1e878d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:48.281050 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5516d8e5-e97c-4d5b-a9a6-5365a1e878d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:48.281319 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5516d8e5-e97c-4d5b-a9a6-5365a1e878d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:48.281956 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5516d8e5-e97c-4d5b-a9a6-5365a1e878d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:48.284058 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5516d8e5-e97c-4d5b-a9a6-5365a1e878d1 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:48.294046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1355/2736] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:31:48.298190 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427eb561-53ab-48b4-b979-c4da45b2a3b9 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:48.309042 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427eb561-53ab-48b4-b979-c4da45b2a3b9 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:48.313674 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa7f29-54a8-4e53-9cd9-054bb973d079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:48.314312 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa7f29-54a8-4e53-9cd9-054bb973d079 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:48.314590 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa7f29-54a8-4e53-9cd9-054bb973d079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:48.314861 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa7f29-54a8-4e53-9cd9-054bb973d079 admin admin] PATH_INFO: `/v3/projects/e69ce85635654b9e98a22a8cab380464/users/beba6736cec24c80b426977c7cbd23d7/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:48.316763 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aa7f29-54a8-4e53-9cd9-054bb973d079 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e69ce85635654b9e98a22a8cab380464, user_id=beba6736cec24c80b426977c7cbd23d7, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:48.319474 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aa7f29-54a8-4e53-9cd9-054bb973d079 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:48.329383 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1382/2737] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:48 2022] PUT /identity/v3/projects/e69ce85635654b9e98a22a8cab380464/users/beba6736cec24c80b426977c7cbd23d7/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:48.339460 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6353088-2cdc-49c0-bd7b-86dc90e7826d None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:48.340016 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6353088-2cdc-49c0-bd7b-86dc90e7826d None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:48.340171 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6353088-2cdc-49c0-bd7b-86dc90e7826d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:48.368899 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-b6353088-2cdc-49c0-bd7b-86dc90e7826d None None] MFA Rules not processed for user `beba6736cec24c80b426977c7cbd23d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:48.370057 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-b6353088-2cdc-49c0-bd7b-86dc90e7826d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:48.438145 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1356/2738] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:31:48 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:48.449542 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa7f29-54a8-4e53-9cd9-054bb973d079 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:48.451272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e8aa7f29-54a8-4e53-9cd9-054bb973d079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:48.483275 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:48.511722 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa7f29-54a8-4e53-9cd9-054bb973d079 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:48.515396 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d140a7d-43ef-4594-881d-32d07ce79161 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:48.516030 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d140a7d-43ef-4594-881d-32d07ce79161 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:48.516313 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d140a7d-43ef-4594-881d-32d07ce79161 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:48.516589 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d140a7d-43ef-4594-881d-32d07ce79161 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:48.526355 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d140a7d-43ef-4594-881d-32d07ce79161 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:48.527814 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d140a7d-43ef-4594-881d-32d07ce79161 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:48.537816 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1383/2739] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:48 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:31:48.604735 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6353088-2cdc-49c0-bd7b-86dc90e7826d None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:48.666574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6353088-2cdc-49c0-bd7b-86dc90e7826d None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:48.671054 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847560ff-514a-459a-b26d-84c8e0f452d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:48.671710 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847560ff-514a-459a-b26d-84c8e0f452d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:48.671993 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847560ff-514a-459a-b26d-84c8e0f452d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:48.672266 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847560ff-514a-459a-b26d-84c8e0f452d7 admin admin] PATH_INFO: `/v3/users/eca0f5fd566b48b8b3b6b21b09306112` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:48.673410 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847560ff-514a-459a-b26d-84c8e0f452d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eca0f5fd566b48b8b3b6b21b09306112)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:48.675336 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847560ff-514a-459a-b26d-84c8e0f452d7 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:48.698531 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:48.724214 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-847560ff-514a-459a-b26d-84c8e0f452d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eca0f5fd566b48b8b3b6b21b09306112'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:48.731511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-847560ff-514a-459a-b26d-84c8e0f452d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eca0f5fd566b48b8b3b6b21b09306112'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:48.732207 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-847560ff-514a-459a-b26d-84c8e0f452d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eca0f5fd566b48b8b3b6b21b09306112'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:48.744382 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-847560ff-514a-459a-b26d-84c8e0f452d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eca0f5fd566b48b8b3b6b21b09306112'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:48.751511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-847560ff-514a-459a-b26d-84c8e0f452d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eca0f5fd566b48b8b3b6b21b09306112'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:48.757702 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1357/2740] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:31:48 2022] DELETE /identity/v3/users/eca0f5fd566b48b8b3b6b21b09306112 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:48.762261 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde6a13-67e0-4599-be24-150384d760c1 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:48.762619 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde6a13-67e0-4599-be24-150384d760c1 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:48.762959 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde6a13-67e0-4599-be24-150384d760c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:48.783793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-ffde6a13-67e0-4599-be24-150384d760c1 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:48.785031 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ffde6a13-67e0-4599-be24-150384d760c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:48.843276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1384/2741] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:31:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:48.854269 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847560ff-514a-459a-b26d-84c8e0f452d7 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:48.855748 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-847560ff-514a-459a-b26d-84c8e0f452d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:48.912570 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847560ff-514a-459a-b26d-84c8e0f452d7 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:48.916277 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c3171-8325-4e98-965f-9cc5bbf972ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:48.916923 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c3171-8325-4e98-965f-9cc5bbf972ac service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:48.917149 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c3171-8325-4e98-965f-9cc5bbf972ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:48.917420 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c3171-8325-4e98-965f-9cc5bbf972ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:48.926423 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c3171-8325-4e98-965f-9cc5bbf972ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:48.927941 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c3171-8325-4e98-965f-9cc5bbf972ac service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:48.937483 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1358/2742] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:31:49.204311 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde6a13-67e0-4599-be24-150384d760c1 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:49.206353 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ffde6a13-67e0-4599-be24-150384d760c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:49.220363 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde6a13-67e0-4599-be24-150384d760c1 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:49.227033 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28770200-83a0-4a41-b9ef-7372299eaca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:49.227837 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28770200-83a0-4a41-b9ef-7372299eaca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:49.228123 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28770200-83a0-4a41-b9ef-7372299eaca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:49.228501 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28770200-83a0-4a41-b9ef-7372299eaca0 admin admin] PATH_INFO: `/v3/projects/651ce115cc4e4ea98017ec042e650681` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:49.229686 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28770200-83a0-4a41-b9ef-7372299eaca0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=651ce115cc4e4ea98017ec042e650681)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:49.231623 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28770200-83a0-4a41-b9ef-7372299eaca0 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:49.280521 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-28770200-83a0-4a41-b9ef-7372299eaca0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '651ce115cc4e4ea98017ec042e650681', 'request_id': 'req-28770200-83a0-4a41-b9ef-7372299eaca0'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:49.285954 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-28770200-83a0-4a41-b9ef-7372299eaca0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '651ce115cc4e4ea98017ec042e650681', 'request_id': 'req-28770200-83a0-4a41-b9ef-7372299eaca0'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:49.292138 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-28770200-83a0-4a41-b9ef-7372299eaca0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '651ce115cc4e4ea98017ec042e650681', 'request_id': 'req-28770200-83a0-4a41-b9ef-7372299eaca0'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:49.305633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-28770200-83a0-4a41-b9ef-7372299eaca0 admin admin] The token cache is being invalidate because project 651ce115cc4e4ea98017ec042e650681 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:31:49.305921 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-28770200-83a0-4a41-b9ef-7372299eaca0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:49.306602 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-28770200-83a0-4a41-b9ef-7372299eaca0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:49.308198 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1385/2743] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:31:49 2022] DELETE /identity/v3/projects/651ce115cc4e4ea98017ec042e650681 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:49.312147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c3171-8325-4e98-965f-9cc5bbf972ac service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:49.322523 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c3171-8325-4e98-965f-9cc5bbf972ac service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:49.379994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e865000b-3b87-44db-a05e-c1fbf2e47969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:49.380658 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865000b-3b87-44db-a05e-c1fbf2e47969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:49.380937 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865000b-3b87-44db-a05e-c1fbf2e47969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:49.381207 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865000b-3b87-44db-a05e-c1fbf2e47969 admin admin] PATH_INFO: `/v3/users/beba6736cec24c80b426977c7cbd23d7` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:49.382317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e865000b-3b87-44db-a05e-c1fbf2e47969 admin admin] RBAC: Authorizing `identity:delete_user(user_id=beba6736cec24c80b426977c7cbd23d7)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:49.384298 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e865000b-3b87-44db-a05e-c1fbf2e47969 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:49.428399 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:49.429967 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-e865000b-3b87-44db-a05e-c1fbf2e47969 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'beba6736cec24c80b426977c7cbd23d7'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:49.437958 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-e865000b-3b87-44db-a05e-c1fbf2e47969 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'beba6736cec24c80b426977c7cbd23d7'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:49.438945 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-e865000b-3b87-44db-a05e-c1fbf2e47969 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'beba6736cec24c80b426977c7cbd23d7'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:49.451592 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-e865000b-3b87-44db-a05e-c1fbf2e47969 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'beba6736cec24c80b426977c7cbd23d7'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:49.458669 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-e865000b-3b87-44db-a05e-c1fbf2e47969 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'beba6736cec24c80b426977c7cbd23d7'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:49.465571 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1359/2744] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:31:49 2022] DELETE /identity/v3/users/beba6736cec24c80b426977c7cbd23d7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:49.740132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28770200-83a0-4a41-b9ef-7372299eaca0 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:49.741566 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-28770200-83a0-4a41-b9ef-7372299eaca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:49.756631 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28770200-83a0-4a41-b9ef-7372299eaca0 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:49.808118 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:49.818296 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47955b9a-3a87-45ac-a0ce-43dc6444ea10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:49.818940 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47955b9a-3a87-45ac-a0ce-43dc6444ea10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:49.819219 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47955b9a-3a87-45ac-a0ce-43dc6444ea10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:49.819494 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47955b9a-3a87-45ac-a0ce-43dc6444ea10 admin admin] PATH_INFO: `/v3/projects/e69ce85635654b9e98a22a8cab380464` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:49.820663 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47955b9a-3a87-45ac-a0ce-43dc6444ea10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e69ce85635654b9e98a22a8cab380464)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:49.822625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47955b9a-3a87-45ac-a0ce-43dc6444ea10 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:49.872440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-47955b9a-3a87-45ac-a0ce-43dc6444ea10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e69ce85635654b9e98a22a8cab380464', 'request_id': 'req-47955b9a-3a87-45ac-a0ce-43dc6444ea10'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:49.877965 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-47955b9a-3a87-45ac-a0ce-43dc6444ea10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e69ce85635654b9e98a22a8cab380464', 'request_id': 'req-47955b9a-3a87-45ac-a0ce-43dc6444ea10'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:49.884061 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-47955b9a-3a87-45ac-a0ce-43dc6444ea10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e69ce85635654b9e98a22a8cab380464', 'request_id': 'req-47955b9a-3a87-45ac-a0ce-43dc6444ea10'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:49.896619 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-47955b9a-3a87-45ac-a0ce-43dc6444ea10 admin admin] The token cache is being invalidate because project e69ce85635654b9e98a22a8cab380464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:31:49.896930 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-47955b9a-3a87-45ac-a0ce-43dc6444ea10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:49.897541 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-47955b9a-3a87-45ac-a0ce-43dc6444ea10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:49.899300 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1386/2745] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:31:49 2022] DELETE /identity/v3/projects/e69ce85635654b9e98a22a8cab380464 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:49.904214 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d2911-6e1a-4f0d-85ab-e2034f1d1595 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:49.904509 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d2911-6e1a-4f0d-85ab-e2034f1d1595 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:49.904785 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d2911-6e1a-4f0d-85ab-e2034f1d1595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:49.926219 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-d04d2911-6e1a-4f0d-85ab-e2034f1d1595 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:49.927411 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-d04d2911-6e1a-4f0d-85ab-e2034f1d1595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:49.985132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1360/2746] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:31:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:49.993478 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5243c0-7bbe-43e1-adb4-85945f2367c7 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:49.993771 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5243c0-7bbe-43e1-adb4-85945f2367c7 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:49.994050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5243c0-7bbe-43e1-adb4-85945f2367c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:50.013701 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-2e5243c0-7bbe-43e1-adb4-85945f2367c7 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:50.014913 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2e5243c0-7bbe-43e1-adb4-85945f2367c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:50.023549 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1387/2747] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:31:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:50.027846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d2911-6e1a-4f0d-85ab-e2034f1d1595 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:50.038303 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d2911-6e1a-4f0d-85ab-e2034f1d1595 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:50.043030 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8179880-0193-484e-9a26-5fbb8f789a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:50.043640 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8179880-0193-484e-9a26-5fbb8f789a8c admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:50.043919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8179880-0193-484e-9a26-5fbb8f789a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:50.044239 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8179880-0193-484e-9a26-5fbb8f789a8c admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:50.044892 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8179880-0193-484e-9a26-5fbb8f789a8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:31:50.045192 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8179880-0193-484e-9a26-5fbb8f789a8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:50.046837 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8179880-0193-484e-9a26-5fbb8f789a8c admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:50.058214 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1361/2748] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:31:50.062572 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5243c0-7bbe-43e1-adb4-85945f2367c7 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:50.071628 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5243c0-7bbe-43e1-adb4-85945f2367c7 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:50.076287 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c9645f-7bdc-47c7-ad29-c87f69dbf9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:50.076875 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9645f-7bdc-47c7-ad29-c87f69dbf9b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:50.077151 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9645f-7bdc-47c7-ad29-c87f69dbf9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:50.077419 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9645f-7bdc-47c7-ad29-c87f69dbf9b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:50.078085 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c9645f-7bdc-47c7-ad29-c87f69dbf9b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:50.080512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c9645f-7bdc-47c7-ad29-c87f69dbf9b8 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:50.107748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1388/2749] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:31:50 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:50.111892 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8179880-0193-484e-9a26-5fbb8f789a8c admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:50.121701 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8179880-0193-484e-9a26-5fbb8f789a8c admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:50.177913 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b6be89-94f3-4e0f-8383-0fe2d9848b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:50.178505 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6be89-94f3-4e0f-8383-0fe2d9848b0e admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:50.178781 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6be89-94f3-4e0f-8383-0fe2d9848b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:50.179062 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6be89-94f3-4e0f-8383-0fe2d9848b0e admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:50.179726 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b6be89-94f3-4e0f-8383-0fe2d9848b0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:50.181635 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b6be89-94f3-4e0f-8383-0fe2d9848b0e admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:50.202086 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1362/2750] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:31:50 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:31:50.206840 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c9645f-7bdc-47c7-ad29-c87f69dbf9b8 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:50.217444 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c9645f-7bdc-47c7-ad29-c87f69dbf9b8 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:50.224004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2155ea1-76cf-4134-85c1-f7cc76c29d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:50.224615 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2155ea1-76cf-4134-85c1-f7cc76c29d82 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:50.224898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2155ea1-76cf-4134-85c1-f7cc76c29d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:50.225173 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2155ea1-76cf-4134-85c1-f7cc76c29d82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:50.225820 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2155ea1-76cf-4134-85c1-f7cc76c29d82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:50.227413 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2155ea1-76cf-4134-85c1-f7cc76c29d82 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:50.238263 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1389/2751] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:50.242324 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b6be89-94f3-4e0f-8383-0fe2d9848b0e admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:50.252710 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b6be89-94f3-4e0f-8383-0fe2d9848b0e admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:50.257438 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99f7855-b99b-4b79-834e-0b3d2cb1949e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:50.258054 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f7855-b99b-4b79-834e-0b3d2cb1949e admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:50.258353 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f7855-b99b-4b79-834e-0b3d2cb1949e admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:50.258620 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f7855-b99b-4b79-834e-0b3d2cb1949e admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:50.259248 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99f7855-b99b-4b79-834e-0b3d2cb1949e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:50.260872 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99f7855-b99b-4b79-834e-0b3d2cb1949e admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:50.271919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1363/2752] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:50.276575 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2155ea1-76cf-4134-85c1-f7cc76c29d82 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:50.287737 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2155ea1-76cf-4134-85c1-f7cc76c29d82 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:50.292580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2c547-1069-463e-b9ee-2f6698e2b9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:50.293226 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2c547-1069-463e-b9ee-2f6698e2b9ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:50.293507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2c547-1069-463e-b9ee-2f6698e2b9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:50.293785 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2c547-1069-463e-b9ee-2f6698e2b9ca admin admin] PATH_INFO: `/v3/projects/9b6c7cbde5aa4dc7a0e166a1e69ca34b/users/6327fd189a114e5da5691c7a17e39e01/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:50.309859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a2c547-1069-463e-b9ee-2f6698e2b9ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b6c7cbde5aa4dc7a0e166a1e69ca34b, user_id=6327fd189a114e5da5691c7a17e39e01, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:50.312582 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a2c547-1069-463e-b9ee-2f6698e2b9ca admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:50.326266 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1390/2753] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:50 2022] PUT /identity/v3/projects/9b6c7cbde5aa4dc7a0e166a1e69ca34b/users/6327fd189a114e5da5691c7a17e39e01/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:50.392542 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac83c30-7625-4838-9d32-2d0b314a26b6 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:50.393785 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac83c30-7625-4838-9d32-2d0b314a26b6 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:50.393785 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac83c30-7625-4838-9d32-2d0b314a26b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:50.421891 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-8ac83c30-7625-4838-9d32-2d0b314a26b6 None None] MFA Rules not processed for user `6327fd189a114e5da5691c7a17e39e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:50.423086 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-8ac83c30-7625-4838-9d32-2d0b314a26b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:50.440947 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:50.487835 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1364/2754] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:31:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:50.492326 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2c547-1069-463e-b9ee-2f6698e2b9ca admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:50.553373 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2c547-1069-463e-b9ee-2f6698e2b9ca admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:50.558159 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1f979-d11e-494e-95e3-7e76c7e2976a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:50.558752 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1f979-d11e-494e-95e3-7e76c7e2976a admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:50.559029 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1f979-d11e-494e-95e3-7e76c7e2976a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:50.559298 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1f979-d11e-494e-95e3-7e76c7e2976a admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:50.559966 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d1f979-d11e-494e-95e3-7e76c7e2976a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:50.561847 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d1f979-d11e-494e-95e3-7e76c7e2976a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:50.587013 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1391/2755] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:31:50 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:50.591229 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac83c30-7625-4838-9d32-2d0b314a26b6 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:50.602251 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac83c30-7625-4838-9d32-2d0b314a26b6 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:50.657166 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3f404-4c03-434a-b795-319a76676a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:50.657761 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3f404-4c03-434a-b795-319a76676a9b admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:50.658048 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3f404-4c03-434a-b795-319a76676a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:50.658782 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3f404-4c03-434a-b795-319a76676a9b admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:50.659419 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b3f404-4c03-434a-b795-319a76676a9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:50.661359 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b3f404-4c03-434a-b795-319a76676a9b admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:50.681989 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1365/2756] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:31:50 2022] POST /identity/v3/users => generated 300 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:31:50.686011 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1f979-d11e-494e-95e3-7e76c7e2976a admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:50.692571 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:50.696246 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1f979-d11e-494e-95e3-7e76c7e2976a admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:50.702977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d19da-33e7-4f64-85f3-6594bcfd4aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:50.703694 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d19da-33e7-4f64-85f3-6594bcfd4aac admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:50.704142 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d19da-33e7-4f64-85f3-6594bcfd4aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:50.704431 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d19da-33e7-4f64-85f3-6594bcfd4aac admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:50.705246 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d19da-33e7-4f64-85f3-6594bcfd4aac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:50.707012 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d19da-33e7-4f64-85f3-6594bcfd4aac admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:50.718161 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1392/2757] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:50.722804 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3f404-4c03-434a-b795-319a76676a9b admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:50.732769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3f404-4c03-434a-b795-319a76676a9b admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:50.738088 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8892424-6327-4f9b-8a4c-4a41859539c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:50.738723 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8892424-6327-4f9b-8a4c-4a41859539c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:50.739000 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8892424-6327-4f9b-8a4c-4a41859539c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:50.739269 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8892424-6327-4f9b-8a4c-4a41859539c9 admin admin] PATH_INFO: `/v3/projects/3799d1e753a24058b5b594dc2d9f3011/users/06a36d3cfe33474db5ae414bf8097c46/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:50.753706 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8892424-6327-4f9b-8a4c-4a41859539c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3799d1e753a24058b5b594dc2d9f3011, user_id=06a36d3cfe33474db5ae414bf8097c46, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:50.756332 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8892424-6327-4f9b-8a4c-4a41859539c9 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:50.769959 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1366/2758] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:50 2022] PUT /identity/v3/projects/3799d1e753a24058b5b594dc2d9f3011/users/06a36d3cfe33474db5ae414bf8097c46/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:50.774250 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d19da-33e7-4f64-85f3-6594bcfd4aac admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:50.785330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d19da-33e7-4f64-85f3-6594bcfd4aac admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:50.840463 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e458c7-6412-49d5-979c-6eaf4404b285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:50.841061 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e458c7-6412-49d5-979c-6eaf4404b285 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:50.841338 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e458c7-6412-49d5-979c-6eaf4404b285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:50.841606 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e458c7-6412-49d5-979c-6eaf4404b285 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:50.842230 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e458c7-6412-49d5-979c-6eaf4404b285 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:50.843855 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e458c7-6412-49d5-979c-6eaf4404b285 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:50.853881 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1393/2759] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:50.858138 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8892424-6327-4f9b-8a4c-4a41859539c9 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:50.868375 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8892424-6327-4f9b-8a4c-4a41859539c9 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:50.873173 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b8dc5-e924-4fd0-ac88-d640bc373f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:50.873886 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b8dc5-e924-4fd0-ac88-d640bc373f79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:50.874155 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b8dc5-e924-4fd0-ac88-d640bc373f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:50.874426 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b8dc5-e924-4fd0-ac88-d640bc373f79 admin admin] PATH_INFO: `/v3/projects/3799d1e753a24058b5b594dc2d9f3011/users/06a36d3cfe33474db5ae414bf8097c46/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:50.876434 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2b8dc5-e924-4fd0-ac88-d640bc373f79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3799d1e753a24058b5b594dc2d9f3011, user_id=06a36d3cfe33474db5ae414bf8097c46, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:50.879143 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2b8dc5-e924-4fd0-ac88-d640bc373f79 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:50.889317 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1367/2760] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:50 2022] PUT /identity/v3/projects/3799d1e753a24058b5b594dc2d9f3011/users/06a36d3cfe33474db5ae414bf8097c46/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:50.900086 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b686de-c4db-4b9b-ae36-5b14573b87c5 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:50.900379 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b686de-c4db-4b9b-ae36-5b14573b87c5 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:50.900659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b686de-c4db-4b9b-ae36-5b14573b87c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:50.928361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-91b686de-c4db-4b9b-ae36-5b14573b87c5 None None] MFA Rules not processed for user `06a36d3cfe33474db5ae414bf8097c46`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:50.929573 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-91b686de-c4db-4b9b-ae36-5b14573b87c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:51.000237 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1394/2761] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:31:50 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:51.010900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b8dc5-e924-4fd0-ac88-d640bc373f79 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:51.012609 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-0c2b8dc5-e924-4fd0-ac88-d640bc373f79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:51.069439 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b8dc5-e924-4fd0-ac88-d640bc373f79 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:51.073203 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1bcb5-be8c-49d9-aebd-ddb30735c821 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:51.073798 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1bcb5-be8c-49d9-aebd-ddb30735c821 service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:51.074066 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1bcb5-be8c-49d9-aebd-ddb30735c821 service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:51.074332 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1bcb5-be8c-49d9-aebd-ddb30735c821 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:51.084517 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba1bcb5-be8c-49d9-aebd-ddb30735c821 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:51.086023 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba1bcb5-be8c-49d9-aebd-ddb30735c821 service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:51.095784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1368/2762] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:51 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:31:51.637445 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b686de-c4db-4b9b-ae36-5b14573b87c5 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:51.699268 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b686de-c4db-4b9b-ae36-5b14573b87c5 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:51.704790 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac53b36-e154-48cc-8767-b959a6f4cfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:51.705446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac53b36-e154-48cc-8767-b959a6f4cfc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:51.705726 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac53b36-e154-48cc-8767-b959a6f4cfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:51.706003 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac53b36-e154-48cc-8767-b959a6f4cfc8 admin admin] PATH_INFO: `/v3/users/6327fd189a114e5da5691c7a17e39e01` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:51.707127 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac53b36-e154-48cc-8767-b959a6f4cfc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6327fd189a114e5da5691c7a17e39e01)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:51.709092 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac53b36-e154-48cc-8767-b959a6f4cfc8 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:51.725936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:51.757819 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-bac53b36-e154-48cc-8767-b959a6f4cfc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6327fd189a114e5da5691c7a17e39e01'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:51.764992 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-bac53b36-e154-48cc-8767-b959a6f4cfc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6327fd189a114e5da5691c7a17e39e01'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:51.765721 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-bac53b36-e154-48cc-8767-b959a6f4cfc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6327fd189a114e5da5691c7a17e39e01'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:51.766422 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-bac53b36-e154-48cc-8767-b959a6f4cfc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6327fd189a114e5da5691c7a17e39e01'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:51.778659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-bac53b36-e154-48cc-8767-b959a6f4cfc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6327fd189a114e5da5691c7a17e39e01'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:51.792394 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1395/2763] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:31:51 2022] DELETE /identity/v3/users/6327fd189a114e5da5691c7a17e39e01 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:51.797248 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ccb83-dfc8-447e-a52c-b2fe858b7e1e None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:51.797754 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ccb83-dfc8-447e-a52c-b2fe858b7e1e None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:51.797887 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ccb83-dfc8-447e-a52c-b2fe858b7e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:51.818109 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-608ccb83-dfc8-447e-a52c-b2fe858b7e1e None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:51.819320 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-608ccb83-dfc8-447e-a52c-b2fe858b7e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:51.827549 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:51.880788 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1369/2764] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:31:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:31:51.891025 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac53b36-e154-48cc-8767-b959a6f4cfc8 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:51.892460 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-bac53b36-e154-48cc-8767-b959a6f4cfc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:51.948758 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac53b36-e154-48cc-8767-b959a6f4cfc8 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:51.952328 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c873b-c739-42c0-8136-7f589b2fe90a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:51.952924 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c873b-c739-42c0-8136-7f589b2fe90a service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:51.953202 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c873b-c739-42c0-8136-7f589b2fe90a service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:51.953472 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c873b-c739-42c0-8136-7f589b2fe90a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:51.962188 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81c873b-c739-42c0-8136-7f589b2fe90a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:51.963621 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81c873b-c739-42c0-8136-7f589b2fe90a service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:51.973196 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1396/2765] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:31:52.238544 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ccb83-dfc8-447e-a52c-b2fe858b7e1e None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:52.240093 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-608ccb83-dfc8-447e-a52c-b2fe858b7e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:52.253866 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ccb83-dfc8-447e-a52c-b2fe858b7e1e None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:52.259875 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e852560-e706-4b2b-87c2-fcaa532557ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:52.260529 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e852560-e706-4b2b-87c2-fcaa532557ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:52.260799 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e852560-e706-4b2b-87c2-fcaa532557ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:52.261071 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e852560-e706-4b2b-87c2-fcaa532557ce admin admin] PATH_INFO: `/v3/projects/9b6c7cbde5aa4dc7a0e166a1e69ca34b` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:52.262212 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e852560-e706-4b2b-87c2-fcaa532557ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b6c7cbde5aa4dc7a0e166a1e69ca34b)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:52.264869 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e852560-e706-4b2b-87c2-fcaa532557ce admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:52.316487 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8e852560-e706-4b2b-87c2-fcaa532557ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b6c7cbde5aa4dc7a0e166a1e69ca34b', 'request_id': 'req-8e852560-e706-4b2b-87c2-fcaa532557ce'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:52.323784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8e852560-e706-4b2b-87c2-fcaa532557ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b6c7cbde5aa4dc7a0e166a1e69ca34b', 'request_id': 'req-8e852560-e706-4b2b-87c2-fcaa532557ce'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:52.329441 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8e852560-e706-4b2b-87c2-fcaa532557ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b6c7cbde5aa4dc7a0e166a1e69ca34b', 'request_id': 'req-8e852560-e706-4b2b-87c2-fcaa532557ce'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:52.343003 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8e852560-e706-4b2b-87c2-fcaa532557ce admin admin] The token cache is being invalidate because project 9b6c7cbde5aa4dc7a0e166a1e69ca34b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:31:52.343291 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8e852560-e706-4b2b-87c2-fcaa532557ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:52.343963 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8e852560-e706-4b2b-87c2-fcaa532557ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:52.345452 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1370/2766] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:31:52 2022] DELETE /identity/v3/projects/9b6c7cbde5aa4dc7a0e166a1e69ca34b => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:52.350214 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c873b-c739-42c0-8136-7f589b2fe90a service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:52.360172 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c873b-c739-42c0-8136-7f589b2fe90a service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:52.417872 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc44a36-b3fc-4ee5-b57a-ca1a50b1bfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:52.418502 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc44a36-b3fc-4ee5-b57a-ca1a50b1bfba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:52.418780 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc44a36-b3fc-4ee5-b57a-ca1a50b1bfba admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:52.419051 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc44a36-b3fc-4ee5-b57a-ca1a50b1bfba admin admin] PATH_INFO: `/v3/users/06a36d3cfe33474db5ae414bf8097c46` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:52.420179 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc44a36-b3fc-4ee5-b57a-ca1a50b1bfba admin admin] RBAC: Authorizing `identity:delete_user(user_id=06a36d3cfe33474db5ae414bf8097c46)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:52.422093 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc44a36-b3fc-4ee5-b57a-ca1a50b1bfba admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:52.464803 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:52.473159 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-dbc44a36-b3fc-4ee5-b57a-ca1a50b1bfba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06a36d3cfe33474db5ae414bf8097c46'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:52.480629 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-dbc44a36-b3fc-4ee5-b57a-ca1a50b1bfba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06a36d3cfe33474db5ae414bf8097c46'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:52.481316 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-dbc44a36-b3fc-4ee5-b57a-ca1a50b1bfba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06a36d3cfe33474db5ae414bf8097c46'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:52.481920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-dbc44a36-b3fc-4ee5-b57a-ca1a50b1bfba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06a36d3cfe33474db5ae414bf8097c46'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:52.493757 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-dbc44a36-b3fc-4ee5-b57a-ca1a50b1bfba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06a36d3cfe33474db5ae414bf8097c46'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:52.506510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1397/2767] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:31:52 2022] DELETE /identity/v3/users/06a36d3cfe33474db5ae414bf8097c46 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:52.774033 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e852560-e706-4b2b-87c2-fcaa532557ce admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:52.775490 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-8e852560-e706-4b2b-87c2-fcaa532557ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:52.789683 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e852560-e706-4b2b-87c2-fcaa532557ce admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:52.848697 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e404e3-8a00-4996-8aa3-13ba1e2c7fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:52.849372 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e404e3-8a00-4996-8aa3-13ba1e2c7fb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:52.849582 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e404e3-8a00-4996-8aa3-13ba1e2c7fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:52.849856 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e404e3-8a00-4996-8aa3-13ba1e2c7fb2 admin admin] PATH_INFO: `/v3/projects/3799d1e753a24058b5b594dc2d9f3011` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:52.850963 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e404e3-8a00-4996-8aa3-13ba1e2c7fb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3799d1e753a24058b5b594dc2d9f3011)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:52.852888 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e404e3-8a00-4996-8aa3-13ba1e2c7fb2 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:52.893912 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:52.903735 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-f1e404e3-8a00-4996-8aa3-13ba1e2c7fb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3799d1e753a24058b5b594dc2d9f3011', 'request_id': 'req-f1e404e3-8a00-4996-8aa3-13ba1e2c7fb2'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:52.912076 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-f1e404e3-8a00-4996-8aa3-13ba1e2c7fb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3799d1e753a24058b5b594dc2d9f3011', 'request_id': 'req-f1e404e3-8a00-4996-8aa3-13ba1e2c7fb2'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:52.918765 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-f1e404e3-8a00-4996-8aa3-13ba1e2c7fb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3799d1e753a24058b5b594dc2d9f3011', 'request_id': 'req-f1e404e3-8a00-4996-8aa3-13ba1e2c7fb2'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:52.930468 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-f1e404e3-8a00-4996-8aa3-13ba1e2c7fb2 admin admin] The token cache is being invalidate because project 3799d1e753a24058b5b594dc2d9f3011 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:31:52.930751 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-f1e404e3-8a00-4996-8aa3-13ba1e2c7fb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:52.931396 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-f1e404e3-8a00-4996-8aa3-13ba1e2c7fb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:52.932879 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1371/2768] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:31:52 2022] DELETE /identity/v3/projects/3799d1e753a24058b5b594dc2d9f3011 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:52.937720 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba27d07-e02e-44a2-8e12-87d54d687b58 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:52.938215 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba27d07-e02e-44a2-8e12-87d54d687b58 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:52.938318 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba27d07-e02e-44a2-8e12-87d54d687b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:52.958991 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-bba27d07-e02e-44a2-8e12-87d54d687b58 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:52.960232 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-bba27d07-e02e-44a2-8e12-87d54d687b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:53.019018 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1398/2769] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:31:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:53.027101 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acaca8d-45ec-46ff-a535-8454d03566d8 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:53.027382 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acaca8d-45ec-46ff-a535-8454d03566d8 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:53.027703 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acaca8d-45ec-46ff-a535-8454d03566d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:53.047315 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-5acaca8d-45ec-46ff-a535-8454d03566d8 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:53.048545 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-5acaca8d-45ec-46ff-a535-8454d03566d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:53.056905 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1372/2770] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:31:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:53.061017 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba27d07-e02e-44a2-8e12-87d54d687b58 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:53.071361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba27d07-e02e-44a2-8e12-87d54d687b58 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:53.076042 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3910564-c774-4a6a-b86e-b1d28f27cffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:53.076532 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3910564-c774-4a6a-b86e-b1d28f27cffe admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:53.076808 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3910564-c774-4a6a-b86e-b1d28f27cffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:53.077078 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3910564-c774-4a6a-b86e-b1d28f27cffe admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:53.077721 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3910564-c774-4a6a-b86e-b1d28f27cffe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:31:53.078015 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3910564-c774-4a6a-b86e-b1d28f27cffe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:53.079628 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3910564-c774-4a6a-b86e-b1d28f27cffe admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:53.091497 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1399/2771] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:31:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:31:53.095838 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acaca8d-45ec-46ff-a535-8454d03566d8 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:53.104898 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acaca8d-45ec-46ff-a535-8454d03566d8 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:53.109594 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c663672-e261-48af-b64f-abdb754803ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:53.110178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c663672-e261-48af-b64f-abdb754803ca admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:53.110450 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c663672-e261-48af-b64f-abdb754803ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:53.110714 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c663672-e261-48af-b64f-abdb754803ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:53.111846 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c663672-e261-48af-b64f-abdb754803ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:53.113711 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c663672-e261-48af-b64f-abdb754803ca admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:53.141174 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1373/2772] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:31:53 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:53.145450 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3910564-c774-4a6a-b86e-b1d28f27cffe admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:53.155565 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3910564-c774-4a6a-b86e-b1d28f27cffe admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:53.211944 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc717e-0be8-44c5-ae4a-f204e2883bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:53.212548 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc717e-0be8-44c5-ae4a-f204e2883bd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:53.212828 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc717e-0be8-44c5-ae4a-f204e2883bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:53.213102 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc717e-0be8-44c5-ae4a-f204e2883bd6 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:53.213727 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc717e-0be8-44c5-ae4a-f204e2883bd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:53.215681 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc717e-0be8-44c5-ae4a-f204e2883bd6 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:53.239097 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1400/2773] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:31:53 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:53.243262 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c663672-e261-48af-b64f-abdb754803ca admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:53.254125 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c663672-e261-48af-b64f-abdb754803ca admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:53.258833 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9aeea-f90b-4d45-b391-412a8323d113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:53.259426 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9aeea-f90b-4d45-b391-412a8323d113 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:53.259732 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9aeea-f90b-4d45-b391-412a8323d113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:53.260004 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9aeea-f90b-4d45-b391-412a8323d113 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:53.260645 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9aeea-f90b-4d45-b391-412a8323d113 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:53.262213 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9aeea-f90b-4d45-b391-412a8323d113 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:53.272367 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1374/2774] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:53.276553 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc717e-0be8-44c5-ae4a-f204e2883bd6 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:53.286585 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc717e-0be8-44c5-ae4a-f204e2883bd6 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:53.291212 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d2ad3-fa99-4bb8-a5fd-fdbb1d999564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:53.291832 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d2ad3-fa99-4bb8-a5fd-fdbb1d999564 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:53.292114 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d2ad3-fa99-4bb8-a5fd-fdbb1d999564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:53.292389 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d2ad3-fa99-4bb8-a5fd-fdbb1d999564 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:53.293024 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d2ad3-fa99-4bb8-a5fd-fdbb1d999564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:53.294611 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d2ad3-fa99-4bb8-a5fd-fdbb1d999564 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:53.305860 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1401/2775] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:53.309826 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9aeea-f90b-4d45-b391-412a8323d113 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:53.320366 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9aeea-f90b-4d45-b391-412a8323d113 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:53.324990 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a2e0b-3eb6-4665-ae5b-281deebac279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:53.325627 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a2e0b-3eb6-4665-ae5b-281deebac279 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:53.325902 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a2e0b-3eb6-4665-ae5b-281deebac279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:53.326174 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a2e0b-3eb6-4665-ae5b-281deebac279 admin admin] PATH_INFO: `/v3/projects/19926a1333734b6588d1aacc4e2e50b9/users/93a7045013df4ed4992cbeb36e716683/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:53.340241 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a2e0b-3eb6-4665-ae5b-281deebac279 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19926a1333734b6588d1aacc4e2e50b9, user_id=93a7045013df4ed4992cbeb36e716683, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:53.342889 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a2e0b-3eb6-4665-ae5b-281deebac279 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:53.355780 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1375/2776] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:53 2022] PUT /identity/v3/projects/19926a1333734b6588d1aacc4e2e50b9/users/93a7045013df4ed4992cbeb36e716683/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:31:53.367788 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb046b3-d5a8-432a-9a48-e101cd68ecf9 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:53.368080 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb046b3-d5a8-432a-9a48-e101cd68ecf9 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:53.368356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb046b3-d5a8-432a-9a48-e101cd68ecf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:53.396189 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-6cb046b3-d5a8-432a-9a48-e101cd68ecf9 None None] MFA Rules not processed for user `93a7045013df4ed4992cbeb36e716683`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:53.397331 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6cb046b3-d5a8-432a-9a48-e101cd68ecf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:53.406643 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:53.445193 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a2e0b-3eb6-4665-ae5b-281deebac279 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:53.447071 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-5c8a2e0b-3eb6-4665-ae5b-281deebac279 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:53.466344 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1402/2777] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:31:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:53.477815 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb046b3-d5a8-432a-9a48-e101cd68ecf9 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:53.479178 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6cb046b3-d5a8-432a-9a48-e101cd68ecf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:53.505669 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a2e0b-3eb6-4665-ae5b-281deebac279 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:53.509690 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90302f5-6baf-4044-b725-ebcf25dd4d07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:53.510190 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90302f5-6baf-4044-b725-ebcf25dd4d07 service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:53.510463 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90302f5-6baf-4044-b725-ebcf25dd4d07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:53.510736 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90302f5-6baf-4044-b725-ebcf25dd4d07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:53.512478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-b90302f5-6baf-4044-b725-ebcf25dd4d07 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:53.538486 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb046b3-d5a8-432a-9a48-e101cd68ecf9 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:53.542271 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bf4882-bdb2-4b68-8255-f091a357b689 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:53.542879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf4882-bdb2-4b68-8255-f091a357b689 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:53.543158 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf4882-bdb2-4b68-8255-f091a357b689 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:53.543434 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf4882-bdb2-4b68-8255-f091a357b689 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:53.552632 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bf4882-bdb2-4b68-8255-f091a357b689 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:53.554065 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bf4882-bdb2-4b68-8255-f091a357b689 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:53.563023 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1403/2778] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:31:53.572154 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90302f5-6baf-4044-b725-ebcf25dd4d07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:53.573594 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90302f5-6baf-4044-b725-ebcf25dd4d07 service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:53.583113 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1376/2779] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:53 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:31:58.514217 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bf4882-bdb2-4b68-8255-f091a357b689 service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:58.514217 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-62bf4882-bdb2-4b68-8255-f091a357b689 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:58.583475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bf4882-bdb2-4b68-8255-f091a357b689 service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:58.588653 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7036e2a6-1ea4-4ea3-b2ed-8bf48042da28 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524bcfffcdfa4790885e0d4cbd66fa80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:58.589322 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036e2a6-1ea4-4ea3-b2ed-8bf48042da28 service placement] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:58.589599 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036e2a6-1ea4-4ea3-b2ed-8bf48042da28 service placement] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:58.589870 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036e2a6-1ea4-4ea3-b2ed-8bf48042da28 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:58.591734 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7036e2a6-1ea4-4ea3-b2ed-8bf48042da28 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:58.605252 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7036e2a6-1ea4-4ea3-b2ed-8bf48042da28 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:58.606840 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7036e2a6-1ea4-4ea3-b2ed-8bf48042da28 service placement] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:58.617751 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1404/2780] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:32:03.578556 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=85,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:32:08.612551 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=80 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:32:10.354865 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90302f5-6baf-4044-b725-ebcf25dd4d07 service cinder] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:10.421003 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90302f5-6baf-4044-b725-ebcf25dd4d07 service cinder] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:10.425472 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a6e34-4e37-4e10-b805-836543a9e07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:10.426085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a6e34-4e37-4e10-b805-836543a9e07f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:10.426879 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a6e34-4e37-4e10-b805-836543a9e07f admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:10.427151 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a6e34-4e37-4e10-b805-836543a9e07f admin admin] PATH_INFO: `/v3/users/93a7045013df4ed4992cbeb36e716683` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:10.428286 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a6e34-4e37-4e10-b805-836543a9e07f admin admin] RBAC: Authorizing `identity:delete_user(user_id=93a7045013df4ed4992cbeb36e716683)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:10.430188 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a6e34-4e37-4e10-b805-836543a9e07f admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:10.477308 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8a3a6e34-4e37-4e10-b805-836543a9e07f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93a7045013df4ed4992cbeb36e716683'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:10.484634 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8a3a6e34-4e37-4e10-b805-836543a9e07f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93a7045013df4ed4992cbeb36e716683'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:10.485281 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8a3a6e34-4e37-4e10-b805-836543a9e07f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93a7045013df4ed4992cbeb36e716683'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:10.497803 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8a3a6e34-4e37-4e10-b805-836543a9e07f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93a7045013df4ed4992cbeb36e716683'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:10.504586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-8a3a6e34-4e37-4e10-b805-836543a9e07f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93a7045013df4ed4992cbeb36e716683'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:10.511112 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1377/2781] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:32:10 2022] DELETE /identity/v3/users/93a7045013df4ed4992cbeb36e716683 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:32:10.515650 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace73b5-da9a-4195-9b9d-435deb60ada2 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:10.516159 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace73b5-da9a-4195-9b9d-435deb60ada2 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:10.516281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace73b5-da9a-4195-9b9d-435deb60ada2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:10.535150 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-1ace73b5-da9a-4195-9b9d-435deb60ada2 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:32:10.536382 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1ace73b5-da9a-4195-9b9d-435deb60ada2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:10.595308 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1405/2782] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:32:10.605743 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a6e34-4e37-4e10-b805-836543a9e07f admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:10.607147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-8a3a6e34-4e37-4e10-b805-836543a9e07f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:10.663917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a6e34-4e37-4e10-b805-836543a9e07f admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:10.667517 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de982c-a896-4504-9190-20e946cd31ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:10.668140 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de982c-a896-4504-9190-20e946cd31ff service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:10.668420 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de982c-a896-4504-9190-20e946cd31ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:10.668694 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de982c-a896-4504-9190-20e946cd31ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:10.678207 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de982c-a896-4504-9190-20e946cd31ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:10.679658 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de982c-a896-4504-9190-20e946cd31ff service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:10.689021 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1378/2783] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:32:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:32:10.966555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace73b5-da9a-4195-9b9d-435deb60ada2 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:10.968112 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1ace73b5-da9a-4195-9b9d-435deb60ada2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:10.978336 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b46a2b-8304-49c7-84e7-da56dd190ec1 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:10.978621 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b46a2b-8304-49c7-84e7-da56dd190ec1 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:10.978902 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b46a2b-8304-49c7-84e7-da56dd190ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:10.986882 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace73b5-da9a-4195-9b9d-435deb60ada2 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:10.993067 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040f9c9-5c8b-4222-9207-f8106ec7b455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:10.993713 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040f9c9-5c8b-4222-9207-f8106ec7b455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:10.993993 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040f9c9-5c8b-4222-9207-f8106ec7b455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:10.994265 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040f9c9-5c8b-4222-9207-f8106ec7b455 admin admin] PATH_INFO: `/v3/projects/19926a1333734b6588d1aacc4e2e50b9` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:10.995405 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e040f9c9-5c8b-4222-9207-f8106ec7b455 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19926a1333734b6588d1aacc4e2e50b9)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:10.997955 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e040f9c9-5c8b-4222-9207-f8106ec7b455 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:11.003890 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-44b46a2b-8304-49c7-84e7-da56dd190ec1 None None] MFA Rules not processed for user `ebd8a00f21344c7990ffe1a7ff2986ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:32:11.005860 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-44b46a2b-8304-49c7-84e7-da56dd190ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:11.024851 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:32:11.051024 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e040f9c9-5c8b-4222-9207-f8106ec7b455 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19926a1333734b6588d1aacc4e2e50b9', 'request_id': 'req-e040f9c9-5c8b-4222-9207-f8106ec7b455'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:11.057182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e040f9c9-5c8b-4222-9207-f8106ec7b455 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19926a1333734b6588d1aacc4e2e50b9', 'request_id': 'req-e040f9c9-5c8b-4222-9207-f8106ec7b455'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:11.063410 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e040f9c9-5c8b-4222-9207-f8106ec7b455 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19926a1333734b6588d1aacc4e2e50b9', 'request_id': 'req-e040f9c9-5c8b-4222-9207-f8106ec7b455'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:11.070377 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1379/2784] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:32:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:32:11.078268 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e040f9c9-5c8b-4222-9207-f8106ec7b455 admin admin] The token cache is being invalidate because project 19926a1333734b6588d1aacc4e2e50b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:32:11.079091 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e040f9c9-5c8b-4222-9207-f8106ec7b455 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:11.079199 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e040f9c9-5c8b-4222-9207-f8106ec7b455 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:11.081302 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1406/2785] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:32:10 2022] DELETE /identity/v3/projects/19926a1333734b6588d1aacc4e2e50b9 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:32:11.086686 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84c99f-f3fb-4774-b9f9-325b31f9dc10 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:11.086852 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84c99f-f3fb-4774-b9f9-325b31f9dc10 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:11.087126 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84c99f-f3fb-4774-b9f9-325b31f9dc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:11.108148 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-5d84c99f-f3fb-4774-b9f9-325b31f9dc10 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:32:11.109354 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-5d84c99f-f3fb-4774-b9f9-325b31f9dc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:11.182427 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1380/2786] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:32:11.190351 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad031f-2a95-4879-a2ea-a9b2d996e50f None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:11.190634 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad031f-2a95-4879-a2ea-a9b2d996e50f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:11.190907 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad031f-2a95-4879-a2ea-a9b2d996e50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:11.211271 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-dcad031f-2a95-4879-a2ea-a9b2d996e50f None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:32:11.212499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-dcad031f-2a95-4879-a2ea-a9b2d996e50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:11.220398 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1407/2787] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:32:11.224953 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84c99f-f3fb-4774-b9f9-325b31f9dc10 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:11.234510 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84c99f-f3fb-4774-b9f9-325b31f9dc10 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:11.239692 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f9ac8-d831-41a6-aa12-30b6dc0770c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:11.240284 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f9ac8-d831-41a6-aa12-30b6dc0770c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:11.240557 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f9ac8-d831-41a6-aa12-30b6dc0770c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:11.240824 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f9ac8-d831-41a6-aa12-30b6dc0770c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:11.241467 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f9ac8-d831-41a6-aa12-30b6dc0770c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:32:11.242196 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f9ac8-d831-41a6-aa12-30b6dc0770c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:11.243803 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f9ac8-d831-41a6-aa12-30b6dc0770c3 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:11.256205 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1381/2788] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:32:11.260512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad031f-2a95-4879-a2ea-a9b2d996e50f None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:11.269780 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad031f-2a95-4879-a2ea-a9b2d996e50f None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:11.274664 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93d971-7eb2-4318-aa6f-802becc98f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:11.275632 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93d971-7eb2-4318-aa6f-802becc98f6a admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:11.275632 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93d971-7eb2-4318-aa6f-802becc98f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:11.275855 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93d971-7eb2-4318-aa6f-802becc98f6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:11.276433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f93d971-7eb2-4318-aa6f-802becc98f6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:11.278977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f93d971-7eb2-4318-aa6f-802becc98f6a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:11.307305 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1408/2789] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:32:11 2022] POST /identity/v3/projects => generated 361 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:32:11.311791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f9ac8-d831-41a6-aa12-30b6dc0770c3 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:11.323009 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f9ac8-d831-41a6-aa12-30b6dc0770c3 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:11.385719 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9d03a-4ca0-4b91-a94c-8c10e1be3b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:11.386378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9d03a-4ca0-4b91-a94c-8c10e1be3b2e admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:11.386545 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9d03a-4ca0-4b91-a94c-8c10e1be3b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:11.386818 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9d03a-4ca0-4b91-a94c-8c10e1be3b2e admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:11.387460 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f9d03a-4ca0-4b91-a94c-8c10e1be3b2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:11.389465 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f9d03a-4ca0-4b91-a94c-8c10e1be3b2e admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:11.413085 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1382/2790] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:32:11 2022] POST /identity/v3/users => generated 281 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:32:11.416986 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93d971-7eb2-4318-aa6f-802becc98f6a admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:11.428201 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93d971-7eb2-4318-aa6f-802becc98f6a admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:11.433806 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a1ef4-a6b8-44c8-b139-845d2df29f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:11.434416 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a1ef4-a6b8-44c8-b139-845d2df29f22 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:11.434694 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a1ef4-a6b8-44c8-b139-845d2df29f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:11.434968 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a1ef4-a6b8-44c8-b139-845d2df29f22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:11.435661 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a1ef4-a6b8-44c8-b139-845d2df29f22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:11.437247 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a1ef4-a6b8-44c8-b139-845d2df29f22 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:11.447794 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1409/2791] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:32:11.451794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9d03a-4ca0-4b91-a94c-8c10e1be3b2e admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:11.461929 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9d03a-4ca0-4b91-a94c-8c10e1be3b2e admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:11.466577 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e271e-0fd4-4275-ade0-3c7bee76a11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:11.467186 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e271e-0fd4-4275-ade0-3c7bee76a11c admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:11.467465 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e271e-0fd4-4275-ade0-3c7bee76a11c admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:11.467789 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e271e-0fd4-4275-ade0-3c7bee76a11c admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:11.468432 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096e271e-0fd4-4275-ade0-3c7bee76a11c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:11.470028 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096e271e-0fd4-4275-ade0-3c7bee76a11c admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:11.481913 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1383/2792] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:32:11.486399 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a1ef4-a6b8-44c8-b139-845d2df29f22 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:11.497831 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a1ef4-a6b8-44c8-b139-845d2df29f22 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:11.502756 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38f1be-ba60-437f-aeb1-fc352b396692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:11.503485 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38f1be-ba60-437f-aeb1-fc352b396692 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:11.503863 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38f1be-ba60-437f-aeb1-fc352b396692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:11.504217 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38f1be-ba60-437f-aeb1-fc352b396692 admin admin] PATH_INFO: `/v3/projects/a61e1a706b614c07805c9509594866ad/users/93970262a19c43d5ae44bb67f1850970/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:11.520523 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e38f1be-ba60-437f-aeb1-fc352b396692 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a61e1a706b614c07805c9509594866ad, user_id=93970262a19c43d5ae44bb67f1850970, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:11.523569 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e38f1be-ba60-437f-aeb1-fc352b396692 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:11.538694 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1410/2793] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:32:11 2022] PUT /identity/v3/projects/a61e1a706b614c07805c9509594866ad/users/93970262a19c43d5ae44bb67f1850970/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:32:11.549777 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a430ce-5faf-4d13-8dc1-01d9d87b50ec None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:11.550065 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a430ce-5faf-4d13-8dc1-01d9d87b50ec None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:11.550340 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a430ce-5faf-4d13-8dc1-01d9d87b50ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:11.579305 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-80a430ce-5faf-4d13-8dc1-01d9d87b50ec None None] MFA Rules not processed for user `93970262a19c43d5ae44bb67f1850970`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:32:11.580564 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-80a430ce-5faf-4d13-8dc1-01d9d87b50ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:11.601805 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:32:11.649794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1384/2794] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:32:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:32:11.685145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38f1be-ba60-437f-aeb1-fc352b396692 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:11.686928 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-0e38f1be-ba60-437f-aeb1-fc352b396692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:11.712786 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:32:11.753399 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38f1be-ba60-437f-aeb1-fc352b396692 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:11.757322 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d59893c-280e-4934-96da-18358f8bcdcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:11.757984 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59893c-280e-4934-96da-18358f8bcdcd service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:11.758294 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59893c-280e-4934-96da-18358f8bcdcd service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:11.759074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59893c-280e-4934-96da-18358f8bcdcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:11.768079 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d59893c-280e-4934-96da-18358f8bcdcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:11.769529 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d59893c-280e-4934-96da-18358f8bcdcd service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:11.780138 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1411/2795] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:32:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:32:11.798879 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a430ce-5faf-4d13-8dc1-01d9d87b50ec None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:11.871084 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a430ce-5faf-4d13-8dc1-01d9d87b50ec None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:11.875941 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf552e9-aa83-423e-bb33-a09b9a00b691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:11.876572 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf552e9-aa83-423e-bb33-a09b9a00b691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:11.876866 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf552e9-aa83-423e-bb33-a09b9a00b691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:11.877140 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf552e9-aa83-423e-bb33-a09b9a00b691 admin admin] PATH_INFO: `/v3/users/93970262a19c43d5ae44bb67f1850970` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:11.878290 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf552e9-aa83-423e-bb33-a09b9a00b691 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93970262a19c43d5ae44bb67f1850970)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:11.880828 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf552e9-aa83-423e-bb33-a09b9a00b691 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:11.933965 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-bdf552e9-aa83-423e-bb33-a09b9a00b691 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93970262a19c43d5ae44bb67f1850970'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:11.943470 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-bdf552e9-aa83-423e-bb33-a09b9a00b691 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93970262a19c43d5ae44bb67f1850970'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:11.944259 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-bdf552e9-aa83-423e-bb33-a09b9a00b691 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93970262a19c43d5ae44bb67f1850970'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:11.956635 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-bdf552e9-aa83-423e-bb33-a09b9a00b691 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93970262a19c43d5ae44bb67f1850970'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:11.963400 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-bdf552e9-aa83-423e-bb33-a09b9a00b691 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93970262a19c43d5ae44bb67f1850970'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:11.969238 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1385/2796] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:32:11 2022] DELETE /identity/v3/users/93970262a19c43d5ae44bb67f1850970 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:32:11.973754 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5509f-762f-4f72-9e12-fcb758801558 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:11.974040 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5509f-762f-4f72-9e12-fcb758801558 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:11.974322 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5509f-762f-4f72-9e12-fcb758801558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:11.994133 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-8ac5509f-762f-4f72-9e12-fcb758801558 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:32:11.995321 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8ac5509f-762f-4f72-9e12-fcb758801558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:12.052874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1412/2797] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:32:12.062731 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf552e9-aa83-423e-bb33-a09b9a00b691 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:12.064184 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-bdf552e9-aa83-423e-bb33-a09b9a00b691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:12.110469 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:32:12.124418 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf552e9-aa83-423e-bb33-a09b9a00b691 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:12.128142 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba6006-ae68-4f71-a72e-1482f75d99a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:12.128745 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba6006-ae68-4f71-a72e-1482f75d99a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:12.129018 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba6006-ae68-4f71-a72e-1482f75d99a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:12.129290 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba6006-ae68-4f71-a72e-1482f75d99a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:12.138676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba6006-ae68-4f71-a72e-1482f75d99a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:12.140194 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba6006-ae68-4f71-a72e-1482f75d99a1 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:12.149597 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1386/2798] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:32:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:32:12.432408 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac5509f-762f-4f72-9e12-fcb758801558 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:12.433904 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8ac5509f-762f-4f72-9e12-fcb758801558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:12.448864 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac5509f-762f-4f72-9e12-fcb758801558 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:12.455180 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e91a48-d942-4ade-9701-a2be1ed7f358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:12.455994 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e91a48-d942-4ade-9701-a2be1ed7f358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:12.456383 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e91a48-d942-4ade-9701-a2be1ed7f358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:12.456783 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e91a48-d942-4ade-9701-a2be1ed7f358 admin admin] PATH_INFO: `/v3/projects/a61e1a706b614c07805c9509594866ad` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:12.458630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e91a48-d942-4ade-9701-a2be1ed7f358 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a61e1a706b614c07805c9509594866ad)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:12.460662 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e91a48-d942-4ade-9701-a2be1ed7f358 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:12.507371 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a8e91a48-d942-4ade-9701-a2be1ed7f358 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a61e1a706b614c07805c9509594866ad', 'request_id': 'req-a8e91a48-d942-4ade-9701-a2be1ed7f358'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:12.512980 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a8e91a48-d942-4ade-9701-a2be1ed7f358 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a61e1a706b614c07805c9509594866ad', 'request_id': 'req-a8e91a48-d942-4ade-9701-a2be1ed7f358'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:12.519408 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a8e91a48-d942-4ade-9701-a2be1ed7f358 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a61e1a706b614c07805c9509594866ad', 'request_id': 'req-a8e91a48-d942-4ade-9701-a2be1ed7f358'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:12.533782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a8e91a48-d942-4ade-9701-a2be1ed7f358 admin admin] The token cache is being invalidate because project a61e1a706b614c07805c9509594866ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:32:12.534422 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a8e91a48-d942-4ade-9701-a2be1ed7f358 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:12.534762 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a8e91a48-d942-4ade-9701-a2be1ed7f358 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:12.536937 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1413/2799] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:32:12 2022] DELETE /identity/v3/projects/a61e1a706b614c07805c9509594866ad => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:32:22.145740 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=7 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:32:22.530295 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:32:29.134742 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84617da5-b767-4a03-8fe4-2f9d55f81263 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:29.135223 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84617da5-b767-4a03-8fe4-2f9d55f81263 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:29.135324 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84617da5-b767-4a03-8fe4-2f9d55f81263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:29.154578 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-84617da5-b767-4a03-8fe4-2f9d55f81263 None None] MFA Rules not processed for user `ee580cfc10dd4536b0709e65e3d23d4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:32:29.155829 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-84617da5-b767-4a03-8fe4-2f9d55f81263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:29.210119 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1387/2800] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:32:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:32:29.784791 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e91a48-d942-4ade-9701-a2be1ed7f358 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:29.786871 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a8e91a48-d942-4ade-9701-a2be1ed7f358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:29.845022 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e91a48-d942-4ade-9701-a2be1ed7f358 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:29.849115 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda979d8-9790-4ffd-99ed-78e7c09b0de9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:29.849718 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda979d8-9790-4ffd-99ed-78e7c09b0de9 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:29.849997 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda979d8-9790-4ffd-99ed-78e7c09b0de9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:29.850268 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda979d8-9790-4ffd-99ed-78e7c09b0de9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:29.852034 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-dda979d8-9790-4ffd-99ed-78e7c09b0de9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:29.861459 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda979d8-9790-4ffd-99ed-78e7c09b0de9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:29.862871 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda979d8-9790-4ffd-99ed-78e7c09b0de9 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:29.870775 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1414/2801] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:32:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:32:34.677389 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84617da5-b767-4a03-8fe4-2f9d55f81263 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:34.686884 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84617da5-b767-4a03-8fe4-2f9d55f81263 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:34.690400 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622ede9-3e3a-4a28-8f29-fca01003dc52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:34.690984 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622ede9-3e3a-4a28-8f29-fca01003dc52 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:34.691823 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622ede9-3e3a-4a28-8f29-fca01003dc52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:34.692102 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622ede9-3e3a-4a28-8f29-fca01003dc52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:34.693946 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-b622ede9-3e3a-4a28-8f29-fca01003dc52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:34.760224 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b622ede9-3e3a-4a28-8f29-fca01003dc52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:34.761696 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b622ede9-3e3a-4a28-8f29-fca01003dc52 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:34.773941 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1388/2802] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:32:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:32:39.867315 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=26 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:32:44.768402 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=55 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:32:45.423225 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda979d8-9790-4ffd-99ed-78e7c09b0de9 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:45.424700 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-dda979d8-9790-4ffd-99ed-78e7c09b0de9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:45.503694 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda979d8-9790-4ffd-99ed-78e7c09b0de9 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:45.509520 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfb9866-7c83-46b3-a49c-a77c4b8c8878 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:45.510158 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfb9866-7c83-46b3-a49c-a77c4b8c8878 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:45.510440 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfb9866-7c83-46b3-a49c-a77c4b8c8878 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:45.510718 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfb9866-7c83-46b3-a49c-a77c4b8c8878 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:45.512556 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7bfb9866-7c83-46b3-a49c-a77c4b8c8878 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:45.526434 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfb9866-7c83-46b3-a49c-a77c4b8c8878 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:45.528163 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfb9866-7c83-46b3-a49c-a77c4b8c8878 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:45.540316 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1415/2803] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:32:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:32:55.534752 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=34 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:34:13.191672 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622ede9-3e3a-4a28-8f29-fca01003dc52 service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:34:13.193070 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-b622ede9-3e3a-4a28-8f29-fca01003dc52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:13.248307 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622ede9-3e3a-4a28-8f29-fca01003dc52 service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:34:13.251890 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed8163-05b8-4490-8bb9-4fb7520ad11f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:34:13.252478 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed8163-05b8-4490-8bb9-4fb7520ad11f service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:34:13.252750 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed8163-05b8-4490-8bb9-4fb7520ad11f service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:34:13.253016 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed8163-05b8-4490-8bb9-4fb7520ad11f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:34:13.254706 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-93ed8163-05b8-4490-8bb9-4fb7520ad11f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:13.268424 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ed8163-05b8-4490-8bb9-4fb7520ad11f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:34:13.269818 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ed8163-05b8-4490-8bb9-4fb7520ad11f service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:34:13.280631 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1389/2804] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:34:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:34:14.690793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfb9866-7c83-46b3-a49c-a77c4b8c8878 service nova] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:34:14.701015 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfb9866-7c83-46b3-a49c-a77c4b8c8878 service nova] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:34:14.704649 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c810c884-5a1c-4daa-926c-69b6cf8feeea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:34:14.705237 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810c884-5a1c-4daa-926c-69b6cf8feeea service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:34:14.705556 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810c884-5a1c-4daa-926c-69b6cf8feeea service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:34:14.705831 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810c884-5a1c-4daa-926c-69b6cf8feeea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:34:14.707622 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c810c884-5a1c-4daa-926c-69b6cf8feeea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:14.720702 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c810c884-5a1c-4daa-926c-69b6cf8feeea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:34:14.722108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c810c884-5a1c-4daa-926c-69b6cf8feeea service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:34:14.732926 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1416/2805] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:34:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:34:23.275961 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=26 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:34:24.728137 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=6 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:34:25.777046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5b009-a80d-4827-8229-0bc1ef432127 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:34:25.777528 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5b009-a80d-4827-8229-0bc1ef432127 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:34:25.777648 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5b009-a80d-4827-8229-0bc1ef432127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:34:25.796553 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-65d5b009-a80d-4827-8229-0bc1ef432127 None None] MFA Rules not processed for user `ee580cfc10dd4536b0709e65e3d23d4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:34:25.797855 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-65d5b009-a80d-4827-8229-0bc1ef432127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:25.805341 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1390/2806] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:34:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:34:28.571580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c810c884-5a1c-4daa-926c-69b6cf8feeea service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:34:28.573659 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c810c884-5a1c-4daa-926c-69b6cf8feeea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:28.584874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c810c884-5a1c-4daa-926c-69b6cf8feeea service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:34:28.589203 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee57800c-0b0f-495b-9b7d-9f69c5a51793 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:34:28.589813 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57800c-0b0f-495b-9b7d-9f69c5a51793 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:34:28.590090 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57800c-0b0f-495b-9b7d-9f69c5a51793 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:34:28.590359 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57800c-0b0f-495b-9b7d-9f69c5a51793 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:34:28.592552 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ee57800c-0b0f-495b-9b7d-9f69c5a51793 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:28.602957 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee57800c-0b0f-495b-9b7d-9f69c5a51793 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:34:28.604511 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee57800c-0b0f-495b-9b7d-9f69c5a51793 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:34:28.615186 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1417/2807] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:34:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:34:35.791556 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:34:38.610318 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=6 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:34:52.126334 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5b009-a80d-4827-8229-0bc1ef432127 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:34:52.136450 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5b009-a80d-4827-8229-0bc1ef432127 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:34:52.140019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eff31b-36b6-46bb-8c94-0424493787f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:34:52.140605 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eff31b-36b6-46bb-8c94-0424493787f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:34:52.140882 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eff31b-36b6-46bb-8c94-0424493787f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:34:52.141152 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eff31b-36b6-46bb-8c94-0424493787f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:34:52.142929 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-44eff31b-36b6-46bb-8c94-0424493787f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:52.205144 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eff31b-36b6-46bb-8c94-0424493787f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:34:52.206481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eff31b-36b6-46bb-8c94-0424493787f6 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:34:52.216374 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1391/2808] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:34:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:34:55.238983 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee57800c-0b0f-495b-9b7d-9f69c5a51793 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:34:55.240506 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ee57800c-0b0f-495b-9b7d-9f69c5a51793 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:55.332345 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee57800c-0b0f-495b-9b7d-9f69c5a51793 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:34:55.337493 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b442e-a1fd-498c-a8ad-c5abbb8be26f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524bcfffcdfa4790885e0d4cbd66fa80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:34:55.338693 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b442e-a1fd-498c-a8ad-c5abbb8be26f service placement] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:34:55.338977 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b442e-a1fd-498c-a8ad-c5abbb8be26f service placement] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:34:55.339253 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b442e-a1fd-498c-a8ad-c5abbb8be26f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:34:55.341042 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1b9b442e-a1fd-498c-a8ad-c5abbb8be26f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:55.354829 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b442e-a1fd-498c-a8ad-c5abbb8be26f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:34:55.356830 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b442e-a1fd-498c-a8ad-c5abbb8be26f service placement] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:34:55.373817 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1418/2809] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:34:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:34:58.143978 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eff31b-36b6-46bb-8c94-0424493787f6 service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:34:58.146426 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-44eff31b-36b6-46bb-8c94-0424493787f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:58.165593 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eff31b-36b6-46bb-8c94-0424493787f6 service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:34:58.172289 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3cabc7-5376-4334-8925-e7309b3e9b98 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524bcfffcdfa4790885e0d4cbd66fa80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:34:58.172907 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3cabc7-5376-4334-8925-e7309b3e9b98 service placement] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:34:58.173384 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3cabc7-5376-4334-8925-e7309b3e9b98 service placement] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:34:58.173528 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3cabc7-5376-4334-8925-e7309b3e9b98 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:34:58.175269 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-df3cabc7-5376-4334-8925-e7309b3e9b98 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:58.191336 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3cabc7-5376-4334-8925-e7309b3e9b98 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:34:58.194578 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3cabc7-5376-4334-8925-e7309b3e9b98 service placement] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:34:58.212108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b442e-a1fd-498c-a8ad-c5abbb8be26f service placement] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:34:58.215336 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1b9b442e-a1fd-498c-a8ad-c5abbb8be26f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:58.219579 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1392/2810] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:34:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:34:58.236806 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b442e-a1fd-498c-a8ad-c5abbb8be26f service placement] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:34:58.251407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676b5ca-b648-4277-a074-b96430f3c7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:34:58.252489 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676b5ca-b648-4277-a074-b96430f3c7b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:34:58.252772 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676b5ca-b648-4277-a074-b96430f3c7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:34:58.253044 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676b5ca-b648-4277-a074-b96430f3c7b0 admin admin] PATH_INFO: `/v3/users/ebd8a00f21344c7990ffe1a7ff2986ef` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:34:58.254344 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676b5ca-b648-4277-a074-b96430f3c7b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebd8a00f21344c7990ffe1a7ff2986ef)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:34:58.256289 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676b5ca-b648-4277-a074-b96430f3c7b0 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:34:58.304141 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2676b5ca-b648-4277-a074-b96430f3c7b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebd8a00f21344c7990ffe1a7ff2986ef'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:34:58.311604 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2676b5ca-b648-4277-a074-b96430f3c7b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebd8a00f21344c7990ffe1a7ff2986ef'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:34:58.312527 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2676b5ca-b648-4277-a074-b96430f3c7b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebd8a00f21344c7990ffe1a7ff2986ef'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:34:58.313210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2676b5ca-b648-4277-a074-b96430f3c7b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebd8a00f21344c7990ffe1a7ff2986ef'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:34:58.325454 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2676b5ca-b648-4277-a074-b96430f3c7b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebd8a00f21344c7990ffe1a7ff2986ef'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:34:58.339208 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1419/2811] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:34:58 2022] DELETE /identity/v3/users/ebd8a00f21344c7990ffe1a7ff2986ef => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:34:58.512329 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3cabc7-5376-4334-8925-e7309b3e9b98 service placement] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:34:58.526309 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3cabc7-5376-4334-8925-e7309b3e9b98 service placement] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:34:58.586720 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9117b-3399-4efc-b1be-82c24965bb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:34:58.587341 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9117b-3399-4efc-b1be-82c24965bb99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:34:58.587649 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9117b-3399-4efc-b1be-82c24965bb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:34:58.588519 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9117b-3399-4efc-b1be-82c24965bb99 admin admin] PATH_INFO: `/v3/projects/4d5cb05cfc60481fa4144ee7b077ca4f` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:34:58.589648 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a9117b-3399-4efc-b1be-82c24965bb99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d5cb05cfc60481fa4144ee7b077ca4f)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:34:58.591587 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a9117b-3399-4efc-b1be-82c24965bb99 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:34:58.638087 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-41a9117b-3399-4efc-b1be-82c24965bb99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d5cb05cfc60481fa4144ee7b077ca4f', 'request_id': 'req-41a9117b-3399-4efc-b1be-82c24965bb99'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:34:58.645779 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-41a9117b-3399-4efc-b1be-82c24965bb99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d5cb05cfc60481fa4144ee7b077ca4f', 'request_id': 'req-41a9117b-3399-4efc-b1be-82c24965bb99'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:34:58.652242 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-41a9117b-3399-4efc-b1be-82c24965bb99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d5cb05cfc60481fa4144ee7b077ca4f', 'request_id': 'req-41a9117b-3399-4efc-b1be-82c24965bb99'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:34:58.663148 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-41a9117b-3399-4efc-b1be-82c24965bb99 admin admin] The token cache is being invalidate because project 4d5cb05cfc60481fa4144ee7b077ca4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:34:58.663431 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-41a9117b-3399-4efc-b1be-82c24965bb99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:34:58.664080 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-41a9117b-3399-4efc-b1be-82c24965bb99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:34:58.665528 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1393/2812] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:34:58 2022] DELETE /identity/v3/projects/4d5cb05cfc60481fa4144ee7b077ca4f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:34:58.669546 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676b5ca-b648-4277-a074-b96430f3c7b0 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:34:58.670984 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2676b5ca-b648-4277-a074-b96430f3c7b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:58.684471 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676b5ca-b648-4277-a074-b96430f3c7b0 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:34:58.740764 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76431467-1993-43ed-99ea-58f9c8272485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:34:58.741389 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76431467-1993-43ed-99ea-58f9c8272485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:34:58.741666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76431467-1993-43ed-99ea-58f9c8272485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:34:58.741936 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76431467-1993-43ed-99ea-58f9c8272485 admin admin] PATH_INFO: `/v3/users/3713e2b332654d49998d42ac8cd482e6` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:34:58.743046 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76431467-1993-43ed-99ea-58f9c8272485 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3713e2b332654d49998d42ac8cd482e6)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:34:58.745021 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76431467-1993-43ed-99ea-58f9c8272485 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:34:58.747250 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:34:58.794565 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-76431467-1993-43ed-99ea-58f9c8272485 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3713e2b332654d49998d42ac8cd482e6'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:34:58.801486 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-76431467-1993-43ed-99ea-58f9c8272485 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3713e2b332654d49998d42ac8cd482e6'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:34:58.802210 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-76431467-1993-43ed-99ea-58f9c8272485 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3713e2b332654d49998d42ac8cd482e6'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:34:58.802845 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-76431467-1993-43ed-99ea-58f9c8272485 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3713e2b332654d49998d42ac8cd482e6'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:34:58.815014 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-76431467-1993-43ed-99ea-58f9c8272485 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3713e2b332654d49998d42ac8cd482e6'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:34:58.827413 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1420/2813] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:34:58 2022] DELETE /identity/v3/users/3713e2b332654d49998d42ac8cd482e6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:34:58.998193 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9117b-3399-4efc-b1be-82c24965bb99 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:34:58.999665 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-41a9117b-3399-4efc-b1be-82c24965bb99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:59.014325 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9117b-3399-4efc-b1be-82c24965bb99 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:34:59.023958 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:34:59.077075 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a96ab-2a64-4eba-854a-0b4c38b38e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:34:59.077704 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a96ab-2a64-4eba-854a-0b4c38b38e16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:34:59.077984 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a96ab-2a64-4eba-854a-0b4c38b38e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:34:59.078261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a96ab-2a64-4eba-854a-0b4c38b38e16 admin admin] PATH_INFO: `/v3/projects/0f6efe7cbc5e4b499f13dc0ccf6a67d3` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:34:59.079379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a96ab-2a64-4eba-854a-0b4c38b38e16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f6efe7cbc5e4b499f13dc0ccf6a67d3)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:34:59.081315 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a96ab-2a64-4eba-854a-0b4c38b38e16 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:34:59.128757 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-587a96ab-2a64-4eba-854a-0b4c38b38e16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f6efe7cbc5e4b499f13dc0ccf6a67d3', 'request_id': 'req-587a96ab-2a64-4eba-854a-0b4c38b38e16'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:34:59.136439 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-587a96ab-2a64-4eba-854a-0b4c38b38e16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f6efe7cbc5e4b499f13dc0ccf6a67d3', 'request_id': 'req-587a96ab-2a64-4eba-854a-0b4c38b38e16'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:34:59.142305 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-587a96ab-2a64-4eba-854a-0b4c38b38e16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f6efe7cbc5e4b499f13dc0ccf6a67d3', 'request_id': 'req-587a96ab-2a64-4eba-854a-0b4c38b38e16'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:34:59.154114 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-587a96ab-2a64-4eba-854a-0b4c38b38e16 admin admin] The token cache is being invalidate because project 0f6efe7cbc5e4b499f13dc0ccf6a67d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:34:59.154746 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-587a96ab-2a64-4eba-854a-0b4c38b38e16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:34:59.155077 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-587a96ab-2a64-4eba-854a-0b4c38b38e16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:34:59.156557 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1394/2814] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:34:58 2022] DELETE /identity/v3/projects/0f6efe7cbc5e4b499f13dc0ccf6a67d3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:35:08.823930 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=25,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:35:09.150519 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:35:56.488152 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76431467-1993-43ed-99ea-58f9c8272485 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:56.489580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-76431467-1993-43ed-99ea-58f9c8272485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:56.561971 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76431467-1993-43ed-99ea-58f9c8272485 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:56.624847 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc48b1-7d31-4bbf-962d-686ae2db0f0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:56.625374 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc48b1-7d31-4bbf-962d-686ae2db0f0d service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:56.625651 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc48b1-7d31-4bbf-962d-686ae2db0f0d service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:56.625922 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc48b1-7d31-4bbf-962d-686ae2db0f0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:56.627670 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-10dc48b1-7d31-4bbf-962d-686ae2db0f0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:56.684095 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc48b1-7d31-4bbf-962d-686ae2db0f0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:56.685720 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc48b1-7d31-4bbf-962d-686ae2db0f0d service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:56.694593 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1421/2815] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:35:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:36:06.691027 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=84 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:36:10.482784 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a96ab-2a64-4eba-854a-0b4c38b38e16 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:10.484372 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-587a96ab-2a64-4eba-854a-0b4c38b38e16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:10.501507 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a96ab-2a64-4eba-854a-0b4c38b38e16 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:10.507390 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db99b1-8fff-4d4d-8339-30d3a4780335 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:10.508217 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db99b1-8fff-4d4d-8339-30d3a4780335 service nova] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:10.508497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db99b1-8fff-4d4d-8339-30d3a4780335 service nova] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:10.508855 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db99b1-8fff-4d4d-8339-30d3a4780335 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:10.511043 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-67db99b1-8fff-4d4d-8339-30d3a4780335 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:10.524577 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db99b1-8fff-4d4d-8339-30d3a4780335 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:10.526248 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db99b1-8fff-4d4d-8339-30d3a4780335 service nova] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:10.538538 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1395/2816] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:36:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:36:13.990114 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce16489-3f72-476e-b5b5-5f5eab66e553 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:13.990619 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce16489-3f72-476e-b5b5-5f5eab66e553 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:13.990735 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce16489-3f72-476e-b5b5-5f5eab66e553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:14.009906 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-5ce16489-3f72-476e-b5b5-5f5eab66e553 None None] MFA Rules not processed for user `ee580cfc10dd4536b0709e65e3d23d4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:36:14.011108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5ce16489-3f72-476e-b5b5-5f5eab66e553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:14.018499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1422/2817] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:36:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:36:14.635549 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db99b1-8fff-4d4d-8339-30d3a4780335 service nova] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:14.636981 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-67db99b1-8fff-4d4d-8339-30d3a4780335 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:14.706722 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db99b1-8fff-4d4d-8339-30d3a4780335 service nova] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:14.711734 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a03e8-9370-4922-a4cd-fd175946fb0c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524bcfffcdfa4790885e0d4cbd66fa80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:14.712346 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a03e8-9370-4922-a4cd-fd175946fb0c service placement] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:14.712609 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a03e8-9370-4922-a4cd-fd175946fb0c service placement] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:14.713065 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a03e8-9370-4922-a4cd-fd175946fb0c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:14.715404 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-527a03e8-9370-4922-a4cd-fd175946fb0c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:14.727824 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527a03e8-9370-4922-a4cd-fd175946fb0c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:14.729485 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527a03e8-9370-4922-a4cd-fd175946fb0c service placement] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:14.739596 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1396/2818] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:36:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:36:24.004510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:36:24.695744 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce16489-3f72-476e-b5b5-5f5eab66e553 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:24.708980 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce16489-3f72-476e-b5b5-5f5eab66e553 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:24.714429 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec532be-e861-427e-a02d-fa191739b38d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:24.715034 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec532be-e861-427e-a02d-fa191739b38d service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:24.715312 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec532be-e861-427e-a02d-fa191739b38d service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:24.715627 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec532be-e861-427e-a02d-fa191739b38d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:24.717468 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9ec532be-e861-427e-a02d-fa191739b38d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:24.731419 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec532be-e861-427e-a02d-fa191739b38d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:24.733108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec532be-e861-427e-a02d-fa191739b38d service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:24.735157 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=38 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:36:24.745476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1423/2819] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:36:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:36:34.739853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=6 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:37:00.474165 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a03e8-9370-4922-a4cd-fd175946fb0c service placement] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:00.535778 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a03e8-9370-4922-a4cd-fd175946fb0c service placement] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:00.541326 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e2e8d-d469-4bed-8674-24354e6c1e34 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524bcfffcdfa4790885e0d4cbd66fa80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:00.541910 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e2e8d-d469-4bed-8674-24354e6c1e34 service placement] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:00.542190 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e2e8d-d469-4bed-8674-24354e6c1e34 service placement] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:00.542458 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e2e8d-d469-4bed-8674-24354e6c1e34 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:00.544242 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-207e2e8d-d469-4bed-8674-24354e6c1e34 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:37:00.577786 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e2e8d-d469-4bed-8674-24354e6c1e34 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:00.579496 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e2e8d-d469-4bed-8674-24354e6c1e34 service placement] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:00.589252 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1397/2820] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:37:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:37:10.585022 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=36 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:01.314448 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec532be-e861-427e-a02d-fa191739b38d service nova] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:01.315893 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9ec532be-e861-427e-a02d-fa191739b38d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:01.421148 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec532be-e861-427e-a02d-fa191739b38d service nova] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:01.424805 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a07e74-3721-4c26-a6b4-e63fea013ed8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:01.425407 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a07e74-3721-4c26-a6b4-e63fea013ed8 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:01.425684 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a07e74-3721-4c26-a6b4-e63fea013ed8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:01.425957 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a07e74-3721-4c26-a6b4-e63fea013ed8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:01.427734 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a5a07e74-3721-4c26-a6b4-e63fea013ed8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:01.510935 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a07e74-3721-4c26-a6b4-e63fea013ed8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:01.513047 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a07e74-3721-4c26-a6b4-e63fea013ed8 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:01.532008 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1424/2821] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:38:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:38:04.621217 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e2e8d-d469-4bed-8674-24354e6c1e34 service placement] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:04.623051 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-207e2e8d-d469-4bed-8674-24354e6c1e34 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:04.637973 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e2e8d-d469-4bed-8674-24354e6c1e34 service placement] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:04.643032 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd01210-fa5c-418b-9e3b-9c0f921f4f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:04.643692 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd01210-fa5c-418b-9e3b-9c0f921f4f36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:04.643975 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd01210-fa5c-418b-9e3b-9c0f921f4f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:04.644249 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd01210-fa5c-418b-9e3b-9c0f921f4f36 admin admin] PATH_INFO: `/v3/users/ee580cfc10dd4536b0709e65e3d23d4d` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:04.645370 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd01210-fa5c-418b-9e3b-9c0f921f4f36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee580cfc10dd4536b0709e65e3d23d4d)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:04.647274 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd01210-fa5c-418b-9e3b-9c0f921f4f36 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:04.694799 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-cbd01210-fa5c-418b-9e3b-9c0f921f4f36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee580cfc10dd4536b0709e65e3d23d4d'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:04.703161 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-cbd01210-fa5c-418b-9e3b-9c0f921f4f36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee580cfc10dd4536b0709e65e3d23d4d'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:04.703808 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-cbd01210-fa5c-418b-9e3b-9c0f921f4f36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee580cfc10dd4536b0709e65e3d23d4d'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:04.716427 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-cbd01210-fa5c-418b-9e3b-9c0f921f4f36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee580cfc10dd4536b0709e65e3d23d4d'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:04.723308 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-cbd01210-fa5c-418b-9e3b-9c0f921f4f36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee580cfc10dd4536b0709e65e3d23d4d'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:04.729149 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1398/2822] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:38:04 2022] DELETE /identity/v3/users/ee580cfc10dd4536b0709e65e3d23d4d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:04.901087 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a07e74-3721-4c26-a6b4-e63fea013ed8 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:04.911165 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a07e74-3721-4c26-a6b4-e63fea013ed8 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:04.942921 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:04.969530 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441620a-af84-4505-8317-ff69451616ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:04.970164 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441620a-af84-4505-8317-ff69451616ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:04.970447 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441620a-af84-4505-8317-ff69451616ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:04.970724 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441620a-af84-4505-8317-ff69451616ee admin admin] PATH_INFO: `/v3/projects/0d3972b5b8b24e999998d2616531f9e7` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:04.971893 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e441620a-af84-4505-8317-ff69451616ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d3972b5b8b24e999998d2616531f9e7)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:04.973870 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e441620a-af84-4505-8317-ff69451616ee admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:05.020480 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e441620a-af84-4505-8317-ff69451616ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d3972b5b8b24e999998d2616531f9e7', 'request_id': 'req-e441620a-af84-4505-8317-ff69451616ee'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:05.025890 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e441620a-af84-4505-8317-ff69451616ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d3972b5b8b24e999998d2616531f9e7', 'request_id': 'req-e441620a-af84-4505-8317-ff69451616ee'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:05.031822 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e441620a-af84-4505-8317-ff69451616ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d3972b5b8b24e999998d2616531f9e7', 'request_id': 'req-e441620a-af84-4505-8317-ff69451616ee'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:05.043579 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e441620a-af84-4505-8317-ff69451616ee admin admin] The token cache is being invalidate because project 0d3972b5b8b24e999998d2616531f9e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:38:05.043865 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e441620a-af84-4505-8317-ff69451616ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:05.044484 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e441620a-af84-4505-8317-ff69451616ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:05.045927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1425/2823] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:38:04 2022] DELETE /identity/v3/projects/0d3972b5b8b24e999998d2616531f9e7 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:05.050901 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832ba8c-aab9-4239-9d3e-c8715437c64e None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:05.051188 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832ba8c-aab9-4239-9d3e-c8715437c64e None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:05.051462 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832ba8c-aab9-4239-9d3e-c8715437c64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:05.108494 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-8832ba8c-aab9-4239-9d3e-c8715437c64e None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:05.109624 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-8832ba8c-aab9-4239-9d3e-c8715437c64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:05.168129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1399/2824] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:38:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:05.176944 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497f946-6353-462d-8aa2-6f5b8b4b9e3e None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:05.177235 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497f946-6353-462d-8aa2-6f5b8b4b9e3e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:05.177512 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497f946-6353-462d-8aa2-6f5b8b4b9e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:05.197450 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-e497f946-6353-462d-8aa2-6f5b8b4b9e3e None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:05.198654 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e497f946-6353-462d-8aa2-6f5b8b4b9e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:05.207002 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1426/2825] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:38:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:05.211173 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832ba8c-aab9-4239-9d3e-c8715437c64e None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:05.220666 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832ba8c-aab9-4239-9d3e-c8715437c64e None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:05.225297 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ba1c4-3020-4bc9-a8b7-5e87aacb1da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:05.225878 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ba1c4-3020-4bc9-a8b7-5e87aacb1da6 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:05.226157 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ba1c4-3020-4bc9-a8b7-5e87aacb1da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:05.226429 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ba1c4-3020-4bc9-a8b7-5e87aacb1da6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:05.227069 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5ba1c4-3020-4bc9-a8b7-5e87aacb1da6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:38:05.227391 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5ba1c4-3020-4bc9-a8b7-5e87aacb1da6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:05.228991 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5ba1c4-3020-4bc9-a8b7-5e87aacb1da6 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:05.240167 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1400/2826] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:38:05.244290 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497f946-6353-462d-8aa2-6f5b8b4b9e3e None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:05.253912 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497f946-6353-462d-8aa2-6f5b8b4b9e3e None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:05.258537 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d328d3-b428-446d-b0ad-d33c8f83340b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:05.259131 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d328d3-b428-446d-b0ad-d33c8f83340b admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:05.259412 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d328d3-b428-446d-b0ad-d33c8f83340b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:05.259838 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d328d3-b428-446d-b0ad-d33c8f83340b admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:05.260480 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d328d3-b428-446d-b0ad-d33c8f83340b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:05.263009 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d328d3-b428-446d-b0ad-d33c8f83340b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:05.289510 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1427/2827] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:38:05 2022] POST /identity/v3/projects => generated 403 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:38:05.294354 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ba1c4-3020-4bc9-a8b7-5e87aacb1da6 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:05.304390 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ba1c4-3020-4bc9-a8b7-5e87aacb1da6 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:05.330669 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:05.364225 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8feefe2-6882-470e-977d-b71939cea867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:05.364754 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8feefe2-6882-470e-977d-b71939cea867 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:05.365031 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8feefe2-6882-470e-977d-b71939cea867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:05.365307 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8feefe2-6882-470e-977d-b71939cea867 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:05.365931 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8feefe2-6882-470e-977d-b71939cea867 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:05.367904 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8feefe2-6882-470e-977d-b71939cea867 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:05.390559 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1401/2828] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:38:05 2022] POST /identity/v3/users => generated 302 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:38:05.395071 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d328d3-b428-446d-b0ad-d33c8f83340b admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:05.405264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d328d3-b428-446d-b0ad-d33c8f83340b admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:05.409855 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7454cc-5a2b-40ee-8b5d-e26f432fc545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:05.410452 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7454cc-5a2b-40ee-8b5d-e26f432fc545 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:05.410731 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7454cc-5a2b-40ee-8b5d-e26f432fc545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:05.411004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7454cc-5a2b-40ee-8b5d-e26f432fc545 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:05.411680 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7454cc-5a2b-40ee-8b5d-e26f432fc545 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:05.413260 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7454cc-5a2b-40ee-8b5d-e26f432fc545 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:05.423898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1428/2829] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:38:05.427986 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8feefe2-6882-470e-977d-b71939cea867 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:05.438405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8feefe2-6882-470e-977d-b71939cea867 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:05.443000 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c18e6-0d48-4857-8b5a-55a09b31bf1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:05.443618 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c18e6-0d48-4857-8b5a-55a09b31bf1a admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:05.443900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c18e6-0d48-4857-8b5a-55a09b31bf1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:05.444173 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c18e6-0d48-4857-8b5a-55a09b31bf1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:05.444806 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8c18e6-0d48-4857-8b5a-55a09b31bf1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:05.446375 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8c18e6-0d48-4857-8b5a-55a09b31bf1a admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:05.456848 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1402/2830] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:38:05.460960 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7454cc-5a2b-40ee-8b5d-e26f432fc545 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:05.471480 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7454cc-5a2b-40ee-8b5d-e26f432fc545 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:05.476592 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d94a57-7055-445b-afac-87365c6544db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:05.477267 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d94a57-7055-445b-afac-87365c6544db admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:05.477558 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d94a57-7055-445b-afac-87365c6544db admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:05.477824 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d94a57-7055-445b-afac-87365c6544db admin admin] PATH_INFO: `/v3/projects/cbc00e8a369d48eea739b99053122b95/users/f6142fd40c3442e7b3384d973fce0f27/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:05.493484 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d94a57-7055-445b-afac-87365c6544db admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbc00e8a369d48eea739b99053122b95, user_id=f6142fd40c3442e7b3384d973fce0f27, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:05.496773 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d94a57-7055-445b-afac-87365c6544db admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:05.512337 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1429/2831] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:38:05 2022] PUT /identity/v3/projects/cbc00e8a369d48eea739b99053122b95/users/f6142fd40c3442e7b3384d973fce0f27/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:05.518252 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c4e75e-b732-4444-9053-6272f00e261a None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:05.518565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c4e75e-b732-4444-9053-6272f00e261a None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:05.518844 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c4e75e-b732-4444-9053-6272f00e261a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:05.538493 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-72c4e75e-b732-4444-9053-6272f00e261a None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:05.539767 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-72c4e75e-b732-4444-9053-6272f00e261a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:05.601182 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1403/2832] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:38:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:05.611961 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d94a57-7055-445b-afac-87365c6544db admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:05.613690 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-03d94a57-7055-445b-afac-87365c6544db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:05.672505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d94a57-7055-445b-afac-87365c6544db admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:05.676051 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae40e86-8f0d-44e3-bcf2-972e1b19420b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:05.676648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae40e86-8f0d-44e3-bcf2-972e1b19420b service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:05.676922 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae40e86-8f0d-44e3-bcf2-972e1b19420b service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:05.677197 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae40e86-8f0d-44e3-bcf2-972e1b19420b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:05.686372 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae40e86-8f0d-44e3-bcf2-972e1b19420b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:05.687820 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae40e86-8f0d-44e3-bcf2-972e1b19420b service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:05.697148 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1430/2833] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:38:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:38:06.313961 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e40030-f9a2-4b2f-ba89-0efc0dde162b None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:06.314497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e40030-f9a2-4b2f-ba89-0efc0dde162b None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:06.314614 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e40030-f9a2-4b2f-ba89-0efc0dde162b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:06.343696 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-a9e40030-f9a2-4b2f-ba89-0efc0dde162b None None] MFA Rules not processed for user `f6142fd40c3442e7b3384d973fce0f27`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:06.344919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-a9e40030-f9a2-4b2f-ba89-0efc0dde162b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:06.413997 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1404/2834] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:38:06 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:06.418433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae40e86-8f0d-44e3-bcf2-972e1b19420b service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:06.423181 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:06.429256 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae40e86-8f0d-44e3-bcf2-972e1b19420b service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:06.435398 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a080a-feb1-4641-a3cd-c9c324865506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:06.436036 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a080a-feb1-4641-a3cd-c9c324865506 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:06.436319 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a080a-feb1-4641-a3cd-c9c324865506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:06.436594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a080a-feb1-4641-a3cd-c9c324865506 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:06.437234 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5a080a-feb1-4641-a3cd-c9c324865506 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:06.439162 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5a080a-feb1-4641-a3cd-c9c324865506 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:06.466588 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1431/2835] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:38:06 2022] POST /identity/v3/projects => generated 405 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:38:06.470301 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e40030-f9a2-4b2f-ba89-0efc0dde162b None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:06.480979 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e40030-f9a2-4b2f-ba89-0efc0dde162b None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:06.519931 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:06.544437 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce24e7-0606-4ca5-8bb8-629376d71736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:06.545034 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce24e7-0606-4ca5-8bb8-629376d71736 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:06.545311 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce24e7-0606-4ca5-8bb8-629376d71736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:06.545581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce24e7-0606-4ca5-8bb8-629376d71736 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:06.546213 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce24e7-0606-4ca5-8bb8-629376d71736 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:06.548155 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce24e7-0606-4ca5-8bb8-629376d71736 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:06.568419 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1405/2836] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:38:06 2022] POST /identity/v3/users => generated 309 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:38:06.572424 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a080a-feb1-4641-a3cd-c9c324865506 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:06.582444 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a080a-feb1-4641-a3cd-c9c324865506 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:06.587043 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0299ef-d938-4fdf-a123-2414f9565304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:06.587694 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0299ef-d938-4fdf-a123-2414f9565304 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:06.587973 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0299ef-d938-4fdf-a123-2414f9565304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:06.588244 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0299ef-d938-4fdf-a123-2414f9565304 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:06.588882 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0299ef-d938-4fdf-a123-2414f9565304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:06.590940 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0299ef-d938-4fdf-a123-2414f9565304 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:06.601240 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1432/2837] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:38:06.606000 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce24e7-0606-4ca5-8bb8-629376d71736 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:06.616050 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce24e7-0606-4ca5-8bb8-629376d71736 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:06.620668 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a9a50-abe7-4e7e-b545-5b995a76de0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:06.621311 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a9a50-abe7-4e7e-b545-5b995a76de0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:06.621632 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a9a50-abe7-4e7e-b545-5b995a76de0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:06.621909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a9a50-abe7-4e7e-b545-5b995a76de0b admin admin] PATH_INFO: `/v3/projects/6dcc6f5c9e6b4e8d9608f1adbf5a6014/users/60d536b825db4754aef401b88c803c29/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:06.636947 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2a9a50-abe7-4e7e-b545-5b995a76de0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dcc6f5c9e6b4e8d9608f1adbf5a6014, user_id=60d536b825db4754aef401b88c803c29, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:06.639617 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2a9a50-abe7-4e7e-b545-5b995a76de0b admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:06.653378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1406/2838] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:38:06 2022] PUT /identity/v3/projects/6dcc6f5c9e6b4e8d9608f1adbf5a6014/users/60d536b825db4754aef401b88c803c29/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:06.657286 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0299ef-d938-4fdf-a123-2414f9565304 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:06.666888 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0299ef-d938-4fdf-a123-2414f9565304 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:06.720765 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c908f-e209-4019-9b98-e3dac41e11cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:06.721285 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c908f-e209-4019-9b98-e3dac41e11cb admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:06.721562 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c908f-e209-4019-9b98-e3dac41e11cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:06.721830 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c908f-e209-4019-9b98-e3dac41e11cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:06.722484 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c908f-e209-4019-9b98-e3dac41e11cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:06.724098 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c908f-e209-4019-9b98-e3dac41e11cb admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:06.734327 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1433/2839] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:38:06.738639 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a9a50-abe7-4e7e-b545-5b995a76de0b admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:06.748765 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a9a50-abe7-4e7e-b545-5b995a76de0b admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:06.753289 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b2e52-20a4-4d78-b249-66f65d11e89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:06.753918 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b2e52-20a4-4d78-b249-66f65d11e89c admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:06.754192 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b2e52-20a4-4d78-b249-66f65d11e89c admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:06.754461 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b2e52-20a4-4d78-b249-66f65d11e89c admin admin] PATH_INFO: `/v3/projects/6dcc6f5c9e6b4e8d9608f1adbf5a6014/users/60d536b825db4754aef401b88c803c29/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:06.756387 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b2e52-20a4-4d78-b249-66f65d11e89c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dcc6f5c9e6b4e8d9608f1adbf5a6014, user_id=60d536b825db4754aef401b88c803c29, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:06.758991 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b2e52-20a4-4d78-b249-66f65d11e89c admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:06.772183 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1407/2840] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:38:06 2022] PUT /identity/v3/projects/6dcc6f5c9e6b4e8d9608f1adbf5a6014/users/60d536b825db4754aef401b88c803c29/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:07.381043 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3b724-fb1e-447d-8138-fa0979ea6542 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:07.381331 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3b724-fb1e-447d-8138-fa0979ea6542 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:07.381607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3b724-fb1e-447d-8138-fa0979ea6542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:07.412236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-52b3b724-fb1e-447d-8138-fa0979ea6542 None None] MFA Rules not processed for user `60d536b825db4754aef401b88c803c29`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:07.413446 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-52b3b724-fb1e-447d-8138-fa0979ea6542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:07.484411 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1434/2841] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:38:07 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:08.736616 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b2e52-20a4-4d78-b249-66f65d11e89c admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:08.798139 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b2e52-20a4-4d78-b249-66f65d11e89c admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:08.802660 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3502c-8294-4ffc-b861-b72da294fa12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:08.803227 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3502c-8294-4ffc-b861-b72da294fa12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:08.803500 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3502c-8294-4ffc-b861-b72da294fa12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:08.803794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3502c-8294-4ffc-b861-b72da294fa12 admin admin] PATH_INFO: `/v3/users/f6142fd40c3442e7b3384d973fce0f27` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:08.804886 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d3502c-8294-4ffc-b861-b72da294fa12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6142fd40c3442e7b3384d973fce0f27)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:08.806782 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d3502c-8294-4ffc-b861-b72da294fa12 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:08.851026 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-78d3502c-8294-4ffc-b861-b72da294fa12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6142fd40c3442e7b3384d973fce0f27'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:08.858603 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-78d3502c-8294-4ffc-b861-b72da294fa12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6142fd40c3442e7b3384d973fce0f27'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:08.859385 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-78d3502c-8294-4ffc-b861-b72da294fa12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6142fd40c3442e7b3384d973fce0f27'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:08.871147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-78d3502c-8294-4ffc-b861-b72da294fa12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6142fd40c3442e7b3384d973fce0f27'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:08.877906 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-78d3502c-8294-4ffc-b861-b72da294fa12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6142fd40c3442e7b3384d973fce0f27'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:08.883738 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1408/2842] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:38:08 2022] DELETE /identity/v3/users/f6142fd40c3442e7b3384d973fce0f27 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:09.055740 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b3b724-fb1e-447d-8138-fa0979ea6542 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:09.066712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b3b724-fb1e-447d-8138-fa0979ea6542 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:09.123331 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:09.133874 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05899a05-0c97-4dc4-95b4-5c0ca67a0b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:09.134499 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05899a05-0c97-4dc4-95b4-5c0ca67a0b3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:09.134775 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05899a05-0c97-4dc4-95b4-5c0ca67a0b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:09.135045 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05899a05-0c97-4dc4-95b4-5c0ca67a0b3a admin admin] PATH_INFO: `/v3/projects/cbc00e8a369d48eea739b99053122b95` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:09.136156 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05899a05-0c97-4dc4-95b4-5c0ca67a0b3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbc00e8a369d48eea739b99053122b95)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:09.138543 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05899a05-0c97-4dc4-95b4-5c0ca67a0b3a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:09.183419 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-05899a05-0c97-4dc4-95b4-5c0ca67a0b3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbc00e8a369d48eea739b99053122b95', 'request_id': 'req-05899a05-0c97-4dc4-95b4-5c0ca67a0b3a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:09.188861 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-05899a05-0c97-4dc4-95b4-5c0ca67a0b3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbc00e8a369d48eea739b99053122b95', 'request_id': 'req-05899a05-0c97-4dc4-95b4-5c0ca67a0b3a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:09.194913 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-05899a05-0c97-4dc4-95b4-5c0ca67a0b3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbc00e8a369d48eea739b99053122b95', 'request_id': 'req-05899a05-0c97-4dc4-95b4-5c0ca67a0b3a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:09.207683 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-05899a05-0c97-4dc4-95b4-5c0ca67a0b3a admin admin] The token cache is being invalidate because project cbc00e8a369d48eea739b99053122b95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:38:09.207967 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-05899a05-0c97-4dc4-95b4-5c0ca67a0b3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:09.208635 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-05899a05-0c97-4dc4-95b4-5c0ca67a0b3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:09.210024 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1435/2843] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:38:09 2022] DELETE /identity/v3/projects/cbc00e8a369d48eea739b99053122b95 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:09.213983 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3502c-8294-4ffc-b861-b72da294fa12 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:09.216075 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-78d3502c-8294-4ffc-b861-b72da294fa12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:09.229497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3502c-8294-4ffc-b861-b72da294fa12 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:09.234096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:09.288814 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf894b4c-db57-489b-963c-799ab431e93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:09.289436 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf894b4c-db57-489b-963c-799ab431e93a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:09.289713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf894b4c-db57-489b-963c-799ab431e93a admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:09.289984 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf894b4c-db57-489b-963c-799ab431e93a admin admin] PATH_INFO: `/v3/users/60d536b825db4754aef401b88c803c29` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:09.291096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf894b4c-db57-489b-963c-799ab431e93a admin admin] RBAC: Authorizing `identity:delete_user(user_id=60d536b825db4754aef401b88c803c29)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:09.293563 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf894b4c-db57-489b-963c-799ab431e93a admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:09.338777 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-cf894b4c-db57-489b-963c-799ab431e93a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60d536b825db4754aef401b88c803c29'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:09.345859 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-cf894b4c-db57-489b-963c-799ab431e93a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60d536b825db4754aef401b88c803c29'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:09.346482 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-cf894b4c-db57-489b-963c-799ab431e93a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60d536b825db4754aef401b88c803c29'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:09.358657 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-cf894b4c-db57-489b-963c-799ab431e93a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60d536b825db4754aef401b88c803c29'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:09.365380 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-cf894b4c-db57-489b-963c-799ab431e93a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60d536b825db4754aef401b88c803c29'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:09.371800 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1409/2844] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:38:09 2022] DELETE /identity/v3/users/60d536b825db4754aef401b88c803c29 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:09.541809 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05899a05-0c97-4dc4-95b4-5c0ca67a0b3a admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:09.543971 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-05899a05-0c97-4dc4-95b4-5c0ca67a0b3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:09.557424 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05899a05-0c97-4dc4-95b4-5c0ca67a0b3a admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:09.614754 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360991d8-e2f2-428e-ae05-93efd9587104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:09.615414 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360991d8-e2f2-428e-ae05-93efd9587104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:09.615721 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360991d8-e2f2-428e-ae05-93efd9587104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:09.615996 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360991d8-e2f2-428e-ae05-93efd9587104 admin admin] PATH_INFO: `/v3/projects/6dcc6f5c9e6b4e8d9608f1adbf5a6014` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:09.617103 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360991d8-e2f2-428e-ae05-93efd9587104 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dcc6f5c9e6b4e8d9608f1adbf5a6014)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:09.619140 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360991d8-e2f2-428e-ae05-93efd9587104 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:09.664500 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-360991d8-e2f2-428e-ae05-93efd9587104 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dcc6f5c9e6b4e8d9608f1adbf5a6014', 'request_id': 'req-360991d8-e2f2-428e-ae05-93efd9587104'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:09.670623 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-360991d8-e2f2-428e-ae05-93efd9587104 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dcc6f5c9e6b4e8d9608f1adbf5a6014', 'request_id': 'req-360991d8-e2f2-428e-ae05-93efd9587104'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:09.676700 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-360991d8-e2f2-428e-ae05-93efd9587104 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dcc6f5c9e6b4e8d9608f1adbf5a6014', 'request_id': 'req-360991d8-e2f2-428e-ae05-93efd9587104'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:09.688579 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-360991d8-e2f2-428e-ae05-93efd9587104 admin admin] The token cache is being invalidate because project 6dcc6f5c9e6b4e8d9608f1adbf5a6014 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:38:09.688866 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-360991d8-e2f2-428e-ae05-93efd9587104 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:09.689517 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-360991d8-e2f2-428e-ae05-93efd9587104 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:09.690982 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1436/2845] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:38:09 2022] DELETE /identity/v3/projects/6dcc6f5c9e6b4e8d9608f1adbf5a6014 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:09.696751 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136da94-2163-4914-886c-fa0b3cbb53bb None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:09.697041 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136da94-2163-4914-886c-fa0b3cbb53bb None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:09.697316 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136da94-2163-4914-886c-fa0b3cbb53bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:09.716976 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-d136da94-2163-4914-886c-fa0b3cbb53bb None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:09.718170 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-d136da94-2163-4914-886c-fa0b3cbb53bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:09.776450 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1410/2846] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:38:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:38:09.785876 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3b40f-2545-43c3-8bd8-8a2cde10975c None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:09.786159 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3b40f-2545-43c3-8bd8-8a2cde10975c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:09.786432 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3b40f-2545-43c3-8bd8-8a2cde10975c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:09.805603 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-89c3b40f-2545-43c3-8bd8-8a2cde10975c None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:09.806785 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-89c3b40f-2545-43c3-8bd8-8a2cde10975c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:09.814431 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1437/2847] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:38:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:09.818396 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d136da94-2163-4914-886c-fa0b3cbb53bb None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:09.828722 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d136da94-2163-4914-886c-fa0b3cbb53bb None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:09.834241 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533a9ea-e662-4b6c-81fe-5d844ebfd772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:09.834895 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533a9ea-e662-4b6c-81fe-5d844ebfd772 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:09.835214 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533a9ea-e662-4b6c-81fe-5d844ebfd772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:09.835601 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533a9ea-e662-4b6c-81fe-5d844ebfd772 admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:09.836449 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9533a9ea-e662-4b6c-81fe-5d844ebfd772 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:38:09.836955 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9533a9ea-e662-4b6c-81fe-5d844ebfd772 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:09.838551 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9533a9ea-e662-4b6c-81fe-5d844ebfd772 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:09.850812 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1411/2848] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:38:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:38:09.855091 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3b40f-2545-43c3-8bd8-8a2cde10975c None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:09.860633 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:09.865073 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3b40f-2545-43c3-8bd8-8a2cde10975c None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:09.870855 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e724e25-dba2-47c1-96ee-1aff37ff5161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:09.871687 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e724e25-dba2-47c1-96ee-1aff37ff5161 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:09.872071 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e724e25-dba2-47c1-96ee-1aff37ff5161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:09.872359 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e724e25-dba2-47c1-96ee-1aff37ff5161 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:09.873175 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e724e25-dba2-47c1-96ee-1aff37ff5161 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:09.875083 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e724e25-dba2-47c1-96ee-1aff37ff5161 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:09.902800 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1438/2849] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:38:09 2022] POST /identity/v3/projects => generated 409 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:38:09.906994 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533a9ea-e662-4b6c-81fe-5d844ebfd772 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:09.917019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533a9ea-e662-4b6c-81fe-5d844ebfd772 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:09.928753 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:09.974652 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902afaa5-eec5-4082-a6e4-3e23e24dca23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:09.975239 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902afaa5-eec5-4082-a6e4-3e23e24dca23 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:09.975513 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902afaa5-eec5-4082-a6e4-3e23e24dca23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:09.975813 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902afaa5-eec5-4082-a6e4-3e23e24dca23 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:09.976445 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902afaa5-eec5-4082-a6e4-3e23e24dca23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:09.978332 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902afaa5-eec5-4082-a6e4-3e23e24dca23 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:10.002757 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1412/2850] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:38:09 2022] POST /identity/v3/users => generated 305 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:38:10.004038 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e724e25-dba2-47c1-96ee-1aff37ff5161 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:10.014789 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e724e25-dba2-47c1-96ee-1aff37ff5161 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:10.019706 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cfcf82-1d5d-4880-8a6a-eddd7e2b1613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:10.020304 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfcf82-1d5d-4880-8a6a-eddd7e2b1613 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:10.020583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfcf82-1d5d-4880-8a6a-eddd7e2b1613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:10.020857 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfcf82-1d5d-4880-8a6a-eddd7e2b1613 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:10.021504 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cfcf82-1d5d-4880-8a6a-eddd7e2b1613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:10.023429 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cfcf82-1d5d-4880-8a6a-eddd7e2b1613 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:10.034301 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1439/2851] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:38:10.038296 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902afaa5-eec5-4082-a6e4-3e23e24dca23 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:10.048244 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902afaa5-eec5-4082-a6e4-3e23e24dca23 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:10.052914 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a26c4-de02-428b-b76c-4a90c9dec9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:10.053497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a26c4-de02-428b-b76c-4a90c9dec9a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:10.053779 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a26c4-de02-428b-b76c-4a90c9dec9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:10.054052 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a26c4-de02-428b-b76c-4a90c9dec9a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:10.054686 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a26c4-de02-428b-b76c-4a90c9dec9a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:10.056313 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a26c4-de02-428b-b76c-4a90c9dec9a5 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:10.066717 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1413/2852] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:38:10.070495 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cfcf82-1d5d-4880-8a6a-eddd7e2b1613 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:10.080999 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cfcf82-1d5d-4880-8a6a-eddd7e2b1613 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:10.085603 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b29233-6c18-457c-9eed-f24f0fa84fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:10.086243 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b29233-6c18-457c-9eed-f24f0fa84fa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:10.086525 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b29233-6c18-457c-9eed-f24f0fa84fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:10.086801 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b29233-6c18-457c-9eed-f24f0fa84fa3 admin admin] PATH_INFO: `/v3/projects/b8c4e7255fe44cc78d0fd4d9f63e9eac/users/56015b50ad9a420eaf3b6fc89ad0b1ab/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:10.101960 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b29233-6c18-457c-9eed-f24f0fa84fa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8c4e7255fe44cc78d0fd4d9f63e9eac, user_id=56015b50ad9a420eaf3b6fc89ad0b1ab, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:10.105238 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b29233-6c18-457c-9eed-f24f0fa84fa3 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:10.118327 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1440/2853] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:38:10 2022] PUT /identity/v3/projects/b8c4e7255fe44cc78d0fd4d9f63e9eac/users/56015b50ad9a420eaf3b6fc89ad0b1ab/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:10.129240 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b5400-f270-4ec0-a1fc-ae28aa09637a None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:10.129527 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b5400-f270-4ec0-a1fc-ae28aa09637a None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:10.129798 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b5400-f270-4ec0-a1fc-ae28aa09637a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:10.157852 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-845b5400-f270-4ec0-a1fc-ae28aa09637a None None] MFA Rules not processed for user `56015b50ad9a420eaf3b6fc89ad0b1ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:10.159041 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-845b5400-f270-4ec0-a1fc-ae28aa09637a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:10.221616 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1414/2854] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:38:10 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:10.226014 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b29233-6c18-457c-9eed-f24f0fa84fa3 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:10.289166 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b29233-6c18-457c-9eed-f24f0fa84fa3 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:10.293776 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a62af7-925d-42ca-928d-73343ba33ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:10.294372 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a62af7-925d-42ca-928d-73343ba33ac9 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:10.294653 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a62af7-925d-42ca-928d-73343ba33ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:10.294927 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a62af7-925d-42ca-928d-73343ba33ac9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:10.295585 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a62af7-925d-42ca-928d-73343ba33ac9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:10.297486 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a62af7-925d-42ca-928d-73343ba33ac9 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:10.323885 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1441/2855] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:38:10 2022] POST /identity/v3/projects => generated 409 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:38:10.328450 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b5400-f270-4ec0-a1fc-ae28aa09637a None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:10.338791 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b5400-f270-4ec0-a1fc-ae28aa09637a None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:10.394903 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdbc769-9751-4021-9ddb-faac03755093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:10.395502 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdbc769-9751-4021-9ddb-faac03755093 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:10.395817 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdbc769-9751-4021-9ddb-faac03755093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:10.396092 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdbc769-9751-4021-9ddb-faac03755093 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:10.396723 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdbc769-9751-4021-9ddb-faac03755093 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:10.398652 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdbc769-9751-4021-9ddb-faac03755093 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:10.422802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1415/2856] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:38:10 2022] POST /identity/v3/users => generated 311 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:38:10.426711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a62af7-925d-42ca-928d-73343ba33ac9 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:10.437550 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a62af7-925d-42ca-928d-73343ba33ac9 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:10.442221 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18525149-4525-452c-8e05-408e6ae2d0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:10.442819 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18525149-4525-452c-8e05-408e6ae2d0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:10.443099 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18525149-4525-452c-8e05-408e6ae2d0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:10.443371 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18525149-4525-452c-8e05-408e6ae2d0e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:10.444052 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18525149-4525-452c-8e05-408e6ae2d0e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:10.445681 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18525149-4525-452c-8e05-408e6ae2d0e6 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:10.456715 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1442/2857] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:38:10.460675 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdbc769-9751-4021-9ddb-faac03755093 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:10.470591 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdbc769-9751-4021-9ddb-faac03755093 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:10.475198 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ef79a-e1c7-4a6b-a56b-54e6e128e3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:10.475865 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ef79a-e1c7-4a6b-a56b-54e6e128e3dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:10.476146 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ef79a-e1c7-4a6b-a56b-54e6e128e3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:10.476421 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ef79a-e1c7-4a6b-a56b-54e6e128e3dc admin admin] PATH_INFO: `/v3/projects/1a5d6054203e4c929a47cceb69be8c50/users/6369cbeeda9f4856b3728444e97d42b6/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:10.488495 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:10.493151 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ef79a-e1c7-4a6b-a56b-54e6e128e3dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a5d6054203e4c929a47cceb69be8c50, user_id=6369cbeeda9f4856b3728444e97d42b6, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:10.497103 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ef79a-e1c7-4a6b-a56b-54e6e128e3dc admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:10.509589 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1416/2858] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:38:10 2022] PUT /identity/v3/projects/1a5d6054203e4c929a47cceb69be8c50/users/6369cbeeda9f4856b3728444e97d42b6/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:10.514157 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18525149-4525-452c-8e05-408e6ae2d0e6 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:10.524583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18525149-4525-452c-8e05-408e6ae2d0e6 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:10.579491 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657f8e2-112a-4de5-aebf-f92d3bd8c97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:10.580054 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657f8e2-112a-4de5-aebf-f92d3bd8c97d admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:10.580335 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657f8e2-112a-4de5-aebf-f92d3bd8c97d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:10.580608 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657f8e2-112a-4de5-aebf-f92d3bd8c97d admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:10.581236 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657f8e2-112a-4de5-aebf-f92d3bd8c97d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:10.582818 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657f8e2-112a-4de5-aebf-f92d3bd8c97d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:10.592309 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1443/2859] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:38:10.596476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ef79a-e1c7-4a6b-a56b-54e6e128e3dc admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:10.607308 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ef79a-e1c7-4a6b-a56b-54e6e128e3dc admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:10.612012 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e98244c-dd19-4c5b-ba52-6eda0e13f345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:10.612655 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98244c-dd19-4c5b-ba52-6eda0e13f345 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:10.612933 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98244c-dd19-4c5b-ba52-6eda0e13f345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:10.613205 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98244c-dd19-4c5b-ba52-6eda0e13f345 admin admin] PATH_INFO: `/v3/projects/1a5d6054203e4c929a47cceb69be8c50/users/6369cbeeda9f4856b3728444e97d42b6/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:10.615132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e98244c-dd19-4c5b-ba52-6eda0e13f345 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a5d6054203e4c929a47cceb69be8c50, user_id=6369cbeeda9f4856b3728444e97d42b6, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:10.617797 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e98244c-dd19-4c5b-ba52-6eda0e13f345 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:10.628743 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1417/2860] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:38:10 2022] PUT /identity/v3/projects/1a5d6054203e4c929a47cceb69be8c50/users/6369cbeeda9f4856b3728444e97d42b6/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:10.639224 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f4ca6-712b-4507-95c9-3d1b567be061 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:10.639511 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f4ca6-712b-4507-95c9-3d1b567be061 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:10.639818 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f4ca6-712b-4507-95c9-3d1b567be061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:10.667173 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-cf1f4ca6-712b-4507-95c9-3d1b567be061 None None] MFA Rules not processed for user `6369cbeeda9f4856b3728444e97d42b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:10.668893 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-cf1f4ca6-712b-4507-95c9-3d1b567be061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:10.673797 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:10.736784 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1444/2861] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:38:10 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:10.748354 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e98244c-dd19-4c5b-ba52-6eda0e13f345 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:10.750046 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-0e98244c-dd19-4c5b-ba52-6eda0e13f345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:10.826780 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e98244c-dd19-4c5b-ba52-6eda0e13f345 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:10.830494 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83626f1-e521-4ab7-bb68-cf0c93e470c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:10.831028 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83626f1-e521-4ab7-bb68-cf0c93e470c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:10.831305 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83626f1-e521-4ab7-bb68-cf0c93e470c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:10.831600 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83626f1-e521-4ab7-bb68-cf0c93e470c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:10.840359 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83626f1-e521-4ab7-bb68-cf0c93e470c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:10.841809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83626f1-e521-4ab7-bb68-cf0c93e470c0 service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:10.850986 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1418/2862] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:38:10 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:38:11.007476 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f4ca6-712b-4507-95c9-3d1b567be061 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:11.017323 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f4ca6-712b-4507-95c9-3d1b567be061 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:11.020897 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a7be2-8555-411b-adb9-d29892a97b5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:11.021530 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a7be2-8555-411b-adb9-d29892a97b5a service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:11.021955 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a7be2-8555-411b-adb9-d29892a97b5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:11.022390 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a7be2-8555-411b-adb9-d29892a97b5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:11.077761 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a7be2-8555-411b-adb9-d29892a97b5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:11.079182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a7be2-8555-411b-adb9-d29892a97b5a service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:11.087963 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1445/2863] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:38:11 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:38:20.615253 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83626f1-e521-4ab7-bb68-cf0c93e470c0 service cinder] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:20.677637 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83626f1-e521-4ab7-bb68-cf0c93e470c0 service cinder] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:20.682324 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e07584-5dec-42b3-a398-efc743cb232b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:20.682966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e07584-5dec-42b3-a398-efc743cb232b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:20.683243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e07584-5dec-42b3-a398-efc743cb232b admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:20.684051 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e07584-5dec-42b3-a398-efc743cb232b admin admin] PATH_INFO: `/v3/users/56015b50ad9a420eaf3b6fc89ad0b1ab` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:20.685166 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e07584-5dec-42b3-a398-efc743cb232b admin admin] RBAC: Authorizing `identity:delete_user(user_id=56015b50ad9a420eaf3b6fc89ad0b1ab)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:20.687096 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e07584-5dec-42b3-a398-efc743cb232b admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:20.732422 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-01e07584-5dec-42b3-a398-efc743cb232b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56015b50ad9a420eaf3b6fc89ad0b1ab'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:20.739428 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-01e07584-5dec-42b3-a398-efc743cb232b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56015b50ad9a420eaf3b6fc89ad0b1ab'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:20.740069 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-01e07584-5dec-42b3-a398-efc743cb232b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56015b50ad9a420eaf3b6fc89ad0b1ab'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:20.745196 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:20.753233 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-01e07584-5dec-42b3-a398-efc743cb232b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56015b50ad9a420eaf3b6fc89ad0b1ab'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:20.760142 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-01e07584-5dec-42b3-a398-efc743cb232b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56015b50ad9a420eaf3b6fc89ad0b1ab'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:20.766447 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1419/2864] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:38:20 2022] DELETE /identity/v3/users/56015b50ad9a420eaf3b6fc89ad0b1ab => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:20.771169 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534b593-9388-441f-9a2d-9d18c6bc23d6 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:20.771662 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534b593-9388-441f-9a2d-9d18c6bc23d6 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:20.771799 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534b593-9388-441f-9a2d-9d18c6bc23d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:20.790470 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-1534b593-9388-441f-9a2d-9d18c6bc23d6 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:20.791706 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1534b593-9388-441f-9a2d-9d18c6bc23d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:20.849686 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1446/2865] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:38:20.859569 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e07584-5dec-42b3-a398-efc743cb232b admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:20.861020 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-01e07584-5dec-42b3-a398-efc743cb232b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:20.916965 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e07584-5dec-42b3-a398-efc743cb232b admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:20.920630 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fdc7f0-5020-4029-a820-2f5bde819937 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:20.921225 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fdc7f0-5020-4029-a820-2f5bde819937 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:20.921501 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fdc7f0-5020-4029-a820-2f5bde819937 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:20.921769 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fdc7f0-5020-4029-a820-2f5bde819937 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:20.930565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fdc7f0-5020-4029-a820-2f5bde819937 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:20.931955 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fdc7f0-5020-4029-a820-2f5bde819937 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:20.941840 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1420/2866] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:38:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:38:21.203841 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1534b593-9388-441f-9a2d-9d18c6bc23d6 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:21.205349 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1534b593-9388-441f-9a2d-9d18c6bc23d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:21.217687 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1534b593-9388-441f-9a2d-9d18c6bc23d6 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:21.222364 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18217c-d809-4104-b812-c557ac6c5b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:21.223063 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18217c-d809-4104-b812-c557ac6c5b58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:21.223432 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18217c-d809-4104-b812-c557ac6c5b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:21.223802 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18217c-d809-4104-b812-c557ac6c5b58 admin admin] PATH_INFO: `/v3/projects/b8c4e7255fe44cc78d0fd4d9f63e9eac` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:21.224984 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d18217c-d809-4104-b812-c557ac6c5b58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8c4e7255fe44cc78d0fd4d9f63e9eac)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:21.227012 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d18217c-d809-4104-b812-c557ac6c5b58 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:21.267745 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:21.277772 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-5d18217c-d809-4104-b812-c557ac6c5b58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8c4e7255fe44cc78d0fd4d9f63e9eac', 'request_id': 'req-5d18217c-d809-4104-b812-c557ac6c5b58'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:21.283243 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-5d18217c-d809-4104-b812-c557ac6c5b58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8c4e7255fe44cc78d0fd4d9f63e9eac', 'request_id': 'req-5d18217c-d809-4104-b812-c557ac6c5b58'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:21.289287 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-5d18217c-d809-4104-b812-c557ac6c5b58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8c4e7255fe44cc78d0fd4d9f63e9eac', 'request_id': 'req-5d18217c-d809-4104-b812-c557ac6c5b58'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:21.301787 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-5d18217c-d809-4104-b812-c557ac6c5b58 admin admin] The token cache is being invalidate because project b8c4e7255fe44cc78d0fd4d9f63e9eac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:38:21.302071 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-5d18217c-d809-4104-b812-c557ac6c5b58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:21.302687 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-5d18217c-d809-4104-b812-c557ac6c5b58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:21.304184 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1447/2867] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:38:21 2022] DELETE /identity/v3/projects/b8c4e7255fe44cc78d0fd4d9f63e9eac => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:21.308189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fdc7f0-5020-4029-a820-2f5bde819937 service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:21.318272 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fdc7f0-5020-4029-a820-2f5bde819937 service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:21.373898 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c70e0-0165-4d43-97c2-31b6d9888b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:21.374893 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c70e0-0165-4d43-97c2-31b6d9888b2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:21.374893 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c70e0-0165-4d43-97c2-31b6d9888b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:21.375104 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c70e0-0165-4d43-97c2-31b6d9888b2b admin admin] PATH_INFO: `/v3/users/6369cbeeda9f4856b3728444e97d42b6` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:21.376169 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22c70e0-0165-4d43-97c2-31b6d9888b2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6369cbeeda9f4856b3728444e97d42b6)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:21.378078 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22c70e0-0165-4d43-97c2-31b6d9888b2b admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:21.423370 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-e22c70e0-0165-4d43-97c2-31b6d9888b2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6369cbeeda9f4856b3728444e97d42b6'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:21.430369 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-e22c70e0-0165-4d43-97c2-31b6d9888b2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6369cbeeda9f4856b3728444e97d42b6'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:21.430980 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-e22c70e0-0165-4d43-97c2-31b6d9888b2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6369cbeeda9f4856b3728444e97d42b6'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:21.442749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-e22c70e0-0165-4d43-97c2-31b6d9888b2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6369cbeeda9f4856b3728444e97d42b6'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:21.449540 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-e22c70e0-0165-4d43-97c2-31b6d9888b2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6369cbeeda9f4856b3728444e97d42b6'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:21.451565 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:21.456341 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1421/2868] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:38:21 2022] DELETE /identity/v3/users/6369cbeeda9f4856b3728444e97d42b6 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:21.731675 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18217c-d809-4104-b812-c557ac6c5b58 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:21.733329 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5d18217c-d809-4104-b812-c557ac6c5b58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:21.748879 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18217c-d809-4104-b812-c557ac6c5b58 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:21.806381 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edab53d-206e-427a-a086-0e40ad517326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:21.807043 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edab53d-206e-427a-a086-0e40ad517326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:21.807276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edab53d-206e-427a-a086-0e40ad517326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:21.807580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edab53d-206e-427a-a086-0e40ad517326 admin admin] PATH_INFO: `/v3/projects/1a5d6054203e4c929a47cceb69be8c50` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:21.808777 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edab53d-206e-427a-a086-0e40ad517326 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a5d6054203e4c929a47cceb69be8c50)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:21.810683 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edab53d-206e-427a-a086-0e40ad517326 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:21.855782 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7edab53d-206e-427a-a086-0e40ad517326 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a5d6054203e4c929a47cceb69be8c50', 'request_id': 'req-7edab53d-206e-427a-a086-0e40ad517326'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:21.861276 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7edab53d-206e-427a-a086-0e40ad517326 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a5d6054203e4c929a47cceb69be8c50', 'request_id': 'req-7edab53d-206e-427a-a086-0e40ad517326'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:21.867293 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7edab53d-206e-427a-a086-0e40ad517326 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a5d6054203e4c929a47cceb69be8c50', 'request_id': 'req-7edab53d-206e-427a-a086-0e40ad517326'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:21.879722 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7edab53d-206e-427a-a086-0e40ad517326 admin admin] The token cache is being invalidate because project 1a5d6054203e4c929a47cceb69be8c50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:38:21.880010 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7edab53d-206e-427a-a086-0e40ad517326 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:21.880626 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7edab53d-206e-427a-a086-0e40ad517326 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:21.882125 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1448/2869] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:38:21 2022] DELETE /identity/v3/projects/1a5d6054203e4c929a47cceb69be8c50 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:21.887612 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57b6a4-3d83-40a4-a940-218e4fc05c01 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:21.888106 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57b6a4-3d83-40a4-a940-218e4fc05c01 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:21.888223 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57b6a4-3d83-40a4-a940-218e4fc05c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:21.908050 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-ea57b6a4-3d83-40a4-a940-218e4fc05c01 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:21.909231 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-ea57b6a4-3d83-40a4-a940-218e4fc05c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:21.966258 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1422/2870] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:38:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:21.974630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b75445-89d4-4dd5-b8d2-d17efcce8969 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:21.974833 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b75445-89d4-4dd5-b8d2-d17efcce8969 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:21.975101 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b75445-89d4-4dd5-b8d2-d17efcce8969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:21.994258 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-35b75445-89d4-4dd5-b8d2-d17efcce8969 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:21.995431 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-35b75445-89d4-4dd5-b8d2-d17efcce8969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:22.004531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1449/2871] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:38:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:22.008777 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57b6a4-3d83-40a4-a940-218e4fc05c01 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:22.019424 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57b6a4-3d83-40a4-a940-218e4fc05c01 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:22.024129 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ddb202-bf81-4116-bff1-88494135da56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:22.024722 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddb202-bf81-4116-bff1-88494135da56 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:22.025002 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddb202-bf81-4116-bff1-88494135da56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:22.025275 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddb202-bf81-4116-bff1-88494135da56 admin admin] PATH_INFO: `/v3/domains` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:22.025946 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ddb202-bf81-4116-bff1-88494135da56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:38:22.026249 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ddb202-bf81-4116-bff1-88494135da56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:22.028645 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ddb202-bf81-4116-bff1-88494135da56 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:22.040153 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1423/2872] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:38:22.044793 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b75445-89d4-4dd5-b8d2-d17efcce8969 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:22.054013 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b75445-89d4-4dd5-b8d2-d17efcce8969 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:22.058704 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5720f5c-5398-4d3b-bbe5-a4039ac5c5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:22.059324 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5720f5c-5398-4d3b-bbe5-a4039ac5c5f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:22.059648 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5720f5c-5398-4d3b-bbe5-a4039ac5c5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:22.059925 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5720f5c-5398-4d3b-bbe5-a4039ac5c5f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:22.060571 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5720f5c-5398-4d3b-bbe5-a4039ac5c5f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:22.062464 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5720f5c-5398-4d3b-bbe5-a4039ac5c5f9 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:22.094628 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1450/2873] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:38:22 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:38:22.095843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ddb202-bf81-4116-bff1-88494135da56 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:22.105732 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ddb202-bf81-4116-bff1-88494135da56 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:22.161966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca9ae9-7c6e-4819-8467-757086de987f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:22.162559 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca9ae9-7c6e-4819-8467-757086de987f admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:22.162840 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca9ae9-7c6e-4819-8467-757086de987f admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:22.163116 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca9ae9-7c6e-4819-8467-757086de987f admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:22.163787 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca9ae9-7c6e-4819-8467-757086de987f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:22.165693 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca9ae9-7c6e-4819-8467-757086de987f admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:22.186960 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1424/2874] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:38:22 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:38:22.191145 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5720f5c-5398-4d3b-bbe5-a4039ac5c5f9 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:22.201931 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5720f5c-5398-4d3b-bbe5-a4039ac5c5f9 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:22.206462 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809c665-64bc-46c4-9fbe-d25a0b8891e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:22.207052 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809c665-64bc-46c4-9fbe-d25a0b8891e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:22.207330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809c665-64bc-46c4-9fbe-d25a0b8891e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:22.207636 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809c665-64bc-46c4-9fbe-d25a0b8891e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:22.208272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c809c665-64bc-46c4-9fbe-d25a0b8891e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:22.209853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c809c665-64bc-46c4-9fbe-d25a0b8891e9 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:22.219511 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1451/2875] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:38:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:38:22.223147 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca9ae9-7c6e-4819-8467-757086de987f admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:22.232880 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca9ae9-7c6e-4819-8467-757086de987f admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:22.237607 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb60383-5945-457d-ae35-ee3910bb5755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:22.238181 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb60383-5945-457d-ae35-ee3910bb5755 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:22.238457 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb60383-5945-457d-ae35-ee3910bb5755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:22.238723 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb60383-5945-457d-ae35-ee3910bb5755 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:22.239350 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb60383-5945-457d-ae35-ee3910bb5755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:22.240966 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb60383-5945-457d-ae35-ee3910bb5755 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:22.252493 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1425/2876] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:38:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:38:22.256662 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809c665-64bc-46c4-9fbe-d25a0b8891e9 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:22.267475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809c665-64bc-46c4-9fbe-d25a0b8891e9 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:22.272166 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d7ea3-63d2-4fe3-9e35-6a9697d1be53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:22.272812 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d7ea3-63d2-4fe3-9e35-6a9697d1be53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:22.273094 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d7ea3-63d2-4fe3-9e35-6a9697d1be53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:22.273396 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d7ea3-63d2-4fe3-9e35-6a9697d1be53 admin admin] PATH_INFO: `/v3/projects/3cce1844e991475d8dd78d53f9f13711/users/59dabf2f5047499dab5bcd924ac88bd2/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:22.285188 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:22.288848 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47d7ea3-63d2-4fe3-9e35-6a9697d1be53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cce1844e991475d8dd78d53f9f13711, user_id=59dabf2f5047499dab5bcd924ac88bd2, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:22.292262 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47d7ea3-63d2-4fe3-9e35-6a9697d1be53 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:22.305735 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1452/2877] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:38:22 2022] PUT /identity/v3/projects/3cce1844e991475d8dd78d53f9f13711/users/59dabf2f5047499dab5bcd924ac88bd2/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:22.316515 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528b73f-0d0e-4aad-9e94-5cedd1d99ee9 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:22.316800 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528b73f-0d0e-4aad-9e94-5cedd1d99ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:22.317075 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528b73f-0d0e-4aad-9e94-5cedd1d99ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:22.346099 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-6528b73f-0d0e-4aad-9e94-5cedd1d99ee9 None None] MFA Rules not processed for user `59dabf2f5047499dab5bcd924ac88bd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:22.347310 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-6528b73f-0d0e-4aad-9e94-5cedd1d99ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:22.402440 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:22.412236 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1426/2878] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:38:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:22.416789 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d7ea3-63d2-4fe3-9e35-6a9697d1be53 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:22.476090 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d7ea3-63d2-4fe3-9e35-6a9697d1be53 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:22.481207 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee754e-ccda-46e4-bfea-7c9428de984c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:22.481797 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee754e-ccda-46e4-bfea-7c9428de984c admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:22.482074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee754e-ccda-46e4-bfea-7c9428de984c admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:22.482346 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee754e-ccda-46e4-bfea-7c9428de984c admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:22.483032 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee754e-ccda-46e4-bfea-7c9428de984c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:22.484902 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee754e-ccda-46e4-bfea-7c9428de984c admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:22.510759 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1453/2879] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:38:22 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:38:22.514780 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528b73f-0d0e-4aad-9e94-5cedd1d99ee9 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:22.525638 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528b73f-0d0e-4aad-9e94-5cedd1d99ee9 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:22.581766 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c559854-f683-4f3b-be2a-b0147cbf9c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:22.582369 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c559854-f683-4f3b-be2a-b0147cbf9c91 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:22.582647 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c559854-f683-4f3b-be2a-b0147cbf9c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:22.582918 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c559854-f683-4f3b-be2a-b0147cbf9c91 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:22.583574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c559854-f683-4f3b-be2a-b0147cbf9c91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:22.585493 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c559854-f683-4f3b-be2a-b0147cbf9c91 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:22.607512 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1427/2880] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:38:22 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:38:22.611406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee754e-ccda-46e4-bfea-7c9428de984c admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:22.622287 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee754e-ccda-46e4-bfea-7c9428de984c admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:22.626954 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1208db-5609-486d-948d-e3c4b7039142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:22.627573 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1208db-5609-486d-948d-e3c4b7039142 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:22.627859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1208db-5609-486d-948d-e3c4b7039142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:22.628133 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1208db-5609-486d-948d-e3c4b7039142 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:22.628772 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1208db-5609-486d-948d-e3c4b7039142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:22.630351 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1208db-5609-486d-948d-e3c4b7039142 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:22.640866 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1454/2881] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:38:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:38:22.644907 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c559854-f683-4f3b-be2a-b0147cbf9c91 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:22.654690 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c559854-f683-4f3b-be2a-b0147cbf9c91 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:22.659311 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa4346-981c-473a-a20c-452118aa83e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:22.659940 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa4346-981c-473a-a20c-452118aa83e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:22.660223 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa4346-981c-473a-a20c-452118aa83e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:22.660495 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa4346-981c-473a-a20c-452118aa83e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:22.661132 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa4346-981c-473a-a20c-452118aa83e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:22.662710 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa4346-981c-473a-a20c-452118aa83e1 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:22.673019 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1428/2882] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:38:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:38:22.677223 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1208db-5609-486d-948d-e3c4b7039142 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:22.687583 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1208db-5609-486d-948d-e3c4b7039142 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:22.692109 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28fd3c-7771-44c4-83f9-97974bfbb558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:22.692741 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28fd3c-7771-44c4-83f9-97974bfbb558 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:22.693021 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28fd3c-7771-44c4-83f9-97974bfbb558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:22.693294 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28fd3c-7771-44c4-83f9-97974bfbb558 admin admin] PATH_INFO: `/v3/projects/45a2f920cce2467fa7103fe8f5412155/users/a95dfe4c96b641468207c87fdfb2121c/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:22.707405 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb28fd3c-7771-44c4-83f9-97974bfbb558 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45a2f920cce2467fa7103fe8f5412155, user_id=a95dfe4c96b641468207c87fdfb2121c, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:22.710081 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb28fd3c-7771-44c4-83f9-97974bfbb558 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:22.722314 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1455/2883] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:38:22 2022] PUT /identity/v3/projects/45a2f920cce2467fa7103fe8f5412155/users/a95dfe4c96b641468207c87fdfb2121c/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:22.788028 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36389f-8e8b-446b-80d6-c4cb5356fcd8 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:22.788310 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36389f-8e8b-446b-80d6-c4cb5356fcd8 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:22.788610 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36389f-8e8b-446b-80d6-c4cb5356fcd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:22.817534 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-ac36389f-8e8b-446b-80d6-c4cb5356fcd8 None None] MFA Rules not processed for user `a95dfe4c96b641468207c87fdfb2121c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:22.818794 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-ac36389f-8e8b-446b-80d6-c4cb5356fcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:22.882364 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1429/2884] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:38:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:22.887022 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28fd3c-7771-44c4-83f9-97974bfbb558 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:22.949069 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28fd3c-7771-44c4-83f9-97974bfbb558 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:22.953645 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b7f29-dbea-4465-915f-077e25366a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:22.954235 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b7f29-dbea-4465-915f-077e25366a75 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:22.954514 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b7f29-dbea-4465-915f-077e25366a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:22.954786 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b7f29-dbea-4465-915f-077e25366a75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:22.955419 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b7f29-dbea-4465-915f-077e25366a75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:22.957416 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b7f29-dbea-4465-915f-077e25366a75 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:22.984369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1456/2885] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:38:22 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:38:22.988435 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac36389f-8e8b-446b-80d6-c4cb5356fcd8 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:22.999493 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac36389f-8e8b-446b-80d6-c4cb5356fcd8 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:23.054523 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618db13b-3818-4db2-aebf-b3e87e524ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:23.055045 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618db13b-3818-4db2-aebf-b3e87e524ad4 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:23.055324 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618db13b-3818-4db2-aebf-b3e87e524ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:23.055623 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618db13b-3818-4db2-aebf-b3e87e524ad4 admin admin] PATH_INFO: `/v3/users` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:23.056243 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618db13b-3818-4db2-aebf-b3e87e524ad4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:23.058155 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618db13b-3818-4db2-aebf-b3e87e524ad4 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:23.068586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:23.080749 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1430/2886] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:38:22 2022] POST /identity/v3/users => generated 298 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:38:23.084788 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b7f29-dbea-4465-915f-077e25366a75 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:23.095111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b7f29-dbea-4465-915f-077e25366a75 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:23.099752 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550a29d-db2d-42d6-9c9e-96e8ddb35a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:23.100346 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550a29d-db2d-42d6-9c9e-96e8ddb35a25 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:23.100625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550a29d-db2d-42d6-9c9e-96e8ddb35a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:23.100898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550a29d-db2d-42d6-9c9e-96e8ddb35a25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:23.101538 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550a29d-db2d-42d6-9c9e-96e8ddb35a25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:23.103121 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550a29d-db2d-42d6-9c9e-96e8ddb35a25 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:23.113689 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1457/2887] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:38:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:38:23.117564 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618db13b-3818-4db2-aebf-b3e87e524ad4 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:23.127692 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618db13b-3818-4db2-aebf-b3e87e524ad4 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:23.132265 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e6585-370f-4b65-9615-ef77fbfd4fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:23.132894 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e6585-370f-4b65-9615-ef77fbfd4fc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:23.133178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e6585-370f-4b65-9615-ef77fbfd4fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:23.133442 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e6585-370f-4b65-9615-ef77fbfd4fc7 admin admin] PATH_INFO: `/v3/projects/63736e0f8ba94e43988e4edb4b3f67df/users/95d86235494a422eaede673e34ed3f61/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:23.149241 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e6585-370f-4b65-9615-ef77fbfd4fc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63736e0f8ba94e43988e4edb4b3f67df, user_id=95d86235494a422eaede673e34ed3f61, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:23.151918 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e6585-370f-4b65-9615-ef77fbfd4fc7 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:23.164341 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1431/2888] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:38:23 2022] PUT /identity/v3/projects/63736e0f8ba94e43988e4edb4b3f67df/users/95d86235494a422eaede673e34ed3f61/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:23.168310 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550a29d-db2d-42d6-9c9e-96e8ddb35a25 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:23.178580 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550a29d-db2d-42d6-9c9e-96e8ddb35a25 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:23.190383 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:23.236122 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76910ea9-a5c4-4da7-a1e6-6aec797f9bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:23.236722 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76910ea9-a5c4-4da7-a1e6-6aec797f9bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:23.237004 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76910ea9-a5c4-4da7-a1e6-6aec797f9bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:23.237279 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76910ea9-a5c4-4da7-a1e6-6aec797f9bb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:23.237910 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76910ea9-a5c4-4da7-a1e6-6aec797f9bb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:23.239554 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76910ea9-a5c4-4da7-a1e6-6aec797f9bb9 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:23.251254 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1458/2889] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:38:23 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:38:23.255531 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e6585-370f-4b65-9615-ef77fbfd4fc7 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:23.266029 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e6585-370f-4b65-9615-ef77fbfd4fc7 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:23.270919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9685d670-723e-4492-a26f-5c9860018730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:23.271587 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9685d670-723e-4492-a26f-5c9860018730 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:23.271875 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9685d670-723e-4492-a26f-5c9860018730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:23.272149 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9685d670-723e-4492-a26f-5c9860018730 admin admin] PATH_INFO: `/v3/projects/63736e0f8ba94e43988e4edb4b3f67df/users/95d86235494a422eaede673e34ed3f61/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:23.274077 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9685d670-723e-4492-a26f-5c9860018730 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63736e0f8ba94e43988e4edb4b3f67df, user_id=95d86235494a422eaede673e34ed3f61, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:23.276858 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9685d670-723e-4492-a26f-5c9860018730 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:23.287223 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1432/2890] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:38:23 2022] PUT /identity/v3/projects/63736e0f8ba94e43988e4edb4b3f67df/users/95d86235494a422eaede673e34ed3f61/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:23.297342 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a68577-ab99-41ee-a3be-2242372e7480 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:23.297631 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a68577-ab99-41ee-a3be-2242372e7480 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:23.297911 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a68577-ab99-41ee-a3be-2242372e7480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:23.325531 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-f0a68577-ab99-41ee-a3be-2242372e7480 None None] MFA Rules not processed for user `95d86235494a422eaede673e34ed3f61`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:23.326678 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f0a68577-ab99-41ee-a3be-2242372e7480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:23.393520 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1459/2891] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:38:23 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:38:23.404810 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9685d670-723e-4492-a26f-5c9860018730 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:23.406518 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-9685d670-723e-4492-a26f-5c9860018730 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:23.463238 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9685d670-723e-4492-a26f-5c9860018730 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:23.466967 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567db24-762c-468b-ac0b-7353162fb8a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:23.468082 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567db24-762c-468b-ac0b-7353162fb8a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:23.468082 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567db24-762c-468b-ac0b-7353162fb8a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:23.468401 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567db24-762c-468b-ac0b-7353162fb8a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:23.524139 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f567db24-762c-468b-ac0b-7353162fb8a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:23.525568 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f567db24-762c-468b-ac0b-7353162fb8a1 service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:23.534431 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1433/2892] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:38:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:38:25.033534 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a68577-ab99-41ee-a3be-2242372e7480 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:25.034952 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f0a68577-ab99-41ee-a3be-2242372e7480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:25.047409 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a68577-ab99-41ee-a3be-2242372e7480 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:25.050937 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8171c-b314-40d2-9c0f-d275ef79acb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:25.051562 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8171c-b314-40d2-9c0f-d275ef79acb8 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:25.051840 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8171c-b314-40d2-9c0f-d275ef79acb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:25.052114 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8171c-b314-40d2-9c0f-d275ef79acb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:25.107593 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8171c-b314-40d2-9c0f-d275ef79acb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:25.108994 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8171c-b314-40d2-9c0f-d275ef79acb8 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:25.117816 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1460/2893] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:38:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:38:25.643649 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567db24-762c-468b-ac0b-7353162fb8a1 service cinder] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:25.652323 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567db24-762c-468b-ac0b-7353162fb8a1 service cinder] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:25.656535 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f75eb9-7e62-46e4-b438-6d1a9851a67a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:25.657127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f75eb9-7e62-46e4-b438-6d1a9851a67a service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:25.657413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f75eb9-7e62-46e4-b438-6d1a9851a67a service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:25.657687 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f75eb9-7e62-46e4-b438-6d1a9851a67a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:25.666509 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f75eb9-7e62-46e4-b438-6d1a9851a67a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:25.667963 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f75eb9-7e62-46e4-b438-6d1a9851a67a service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:25.677191 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1434/2894] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:38:25 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:38:29.532108 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8171c-b314-40d2-9c0f-d275ef79acb8 service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:29.571724 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:29.597116 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8171c-b314-40d2-9c0f-d275ef79acb8 service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:29.601537 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d1cd8-55ce-4cfa-9eef-83fd76c7f2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:29.602148 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d1cd8-55ce-4cfa-9eef-83fd76c7f2ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:29.602422 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d1cd8-55ce-4cfa-9eef-83fd76c7f2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:29.602689 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d1cd8-55ce-4cfa-9eef-83fd76c7f2ef admin admin] PATH_INFO: `/v3/users/59dabf2f5047499dab5bcd924ac88bd2` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:29.603826 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d1cd8-55ce-4cfa-9eef-83fd76c7f2ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=59dabf2f5047499dab5bcd924ac88bd2)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:29.605764 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d1cd8-55ce-4cfa-9eef-83fd76c7f2ef admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:29.651423 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-248d1cd8-55ce-4cfa-9eef-83fd76c7f2ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59dabf2f5047499dab5bcd924ac88bd2'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:29.658569 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-248d1cd8-55ce-4cfa-9eef-83fd76c7f2ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59dabf2f5047499dab5bcd924ac88bd2'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:29.659213 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-248d1cd8-55ce-4cfa-9eef-83fd76c7f2ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59dabf2f5047499dab5bcd924ac88bd2'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:29.659837 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-248d1cd8-55ce-4cfa-9eef-83fd76c7f2ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59dabf2f5047499dab5bcd924ac88bd2'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:29.671980 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-248d1cd8-55ce-4cfa-9eef-83fd76c7f2ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59dabf2f5047499dab5bcd924ac88bd2'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:29.684270 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1461/2895] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:38:29 2022] DELETE /identity/v3/users/59dabf2f5047499dab5bcd924ac88bd2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:29.688792 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f3298-049a-4e64-8180-deba3e1b8b45 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:29.689280 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f3298-049a-4e64-8180-deba3e1b8b45 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:29.689384 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f3298-049a-4e64-8180-deba3e1b8b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:29.708101 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-cd7f3298-049a-4e64-8180-deba3e1b8b45 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:29.709295 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-cd7f3298-049a-4e64-8180-deba3e1b8b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:29.759583 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:29.770260 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1435/2896] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:38:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:29.780322 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d1cd8-55ce-4cfa-9eef-83fd76c7f2ef admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:29.781714 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-248d1cd8-55ce-4cfa-9eef-83fd76c7f2ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:29.836839 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d1cd8-55ce-4cfa-9eef-83fd76c7f2ef admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:29.840535 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba02895-3f73-4cfd-9d96-76dcf4c3ecaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:29.841131 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba02895-3f73-4cfd-9d96-76dcf4c3ecaa service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:29.841409 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba02895-3f73-4cfd-9d96-76dcf4c3ecaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:29.841685 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba02895-3f73-4cfd-9d96-76dcf4c3ecaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:29.851296 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba02895-3f73-4cfd-9d96-76dcf4c3ecaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:29.852748 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba02895-3f73-4cfd-9d96-76dcf4c3ecaa service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:29.862523 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1462/2897] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:38:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:38:30.122004 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f3298-049a-4e64-8180-deba3e1b8b45 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:30.123497 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-cd7f3298-049a-4e64-8180-deba3e1b8b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:30.135439 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f3298-049a-4e64-8180-deba3e1b8b45 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:30.141204 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e991f8-fc67-4513-a3b0-44cde88cd823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:30.142626 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e991f8-fc67-4513-a3b0-44cde88cd823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:30.142976 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e991f8-fc67-4513-a3b0-44cde88cd823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:30.143256 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e991f8-fc67-4513-a3b0-44cde88cd823 admin admin] PATH_INFO: `/v3/projects/3cce1844e991475d8dd78d53f9f13711` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:30.144530 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e991f8-fc67-4513-a3b0-44cde88cd823 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cce1844e991475d8dd78d53f9f13711)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:30.146736 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e991f8-fc67-4513-a3b0-44cde88cd823 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:30.193464 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-88e991f8-fc67-4513-a3b0-44cde88cd823 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cce1844e991475d8dd78d53f9f13711', 'request_id': 'req-88e991f8-fc67-4513-a3b0-44cde88cd823'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:30.200141 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-88e991f8-fc67-4513-a3b0-44cde88cd823 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cce1844e991475d8dd78d53f9f13711', 'request_id': 'req-88e991f8-fc67-4513-a3b0-44cde88cd823'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:30.206113 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-88e991f8-fc67-4513-a3b0-44cde88cd823 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cce1844e991475d8dd78d53f9f13711', 'request_id': 'req-88e991f8-fc67-4513-a3b0-44cde88cd823'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:30.216917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-88e991f8-fc67-4513-a3b0-44cde88cd823 admin admin] The token cache is being invalidate because project 3cce1844e991475d8dd78d53f9f13711 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:38:30.217806 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-88e991f8-fc67-4513-a3b0-44cde88cd823 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:30.218475 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-88e991f8-fc67-4513-a3b0-44cde88cd823 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:30.219969 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1436/2898] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:38:30 2022] DELETE /identity/v3/projects/3cce1844e991475d8dd78d53f9f13711 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:30.223873 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba02895-3f73-4cfd-9d96-76dcf4c3ecaa service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:30.233300 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba02895-3f73-4cfd-9d96-76dcf4c3ecaa service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:30.284016 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:30.291183 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6621c-ff82-427c-81e1-f3080c6dd675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:30.292092 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6621c-ff82-427c-81e1-f3080c6dd675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:30.292625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6621c-ff82-427c-81e1-f3080c6dd675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:30.292922 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6621c-ff82-427c-81e1-f3080c6dd675 admin admin] PATH_INFO: `/v3/users/a95dfe4c96b641468207c87fdfb2121c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:30.294268 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba6621c-ff82-427c-81e1-f3080c6dd675 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a95dfe4c96b641468207c87fdfb2121c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:30.296263 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba6621c-ff82-427c-81e1-f3080c6dd675 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:30.342335 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-eba6621c-ff82-427c-81e1-f3080c6dd675 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a95dfe4c96b641468207c87fdfb2121c'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:30.349189 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-eba6621c-ff82-427c-81e1-f3080c6dd675 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a95dfe4c96b641468207c87fdfb2121c'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:30.349830 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-eba6621c-ff82-427c-81e1-f3080c6dd675 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a95dfe4c96b641468207c87fdfb2121c'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:30.350455 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-eba6621c-ff82-427c-81e1-f3080c6dd675 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a95dfe4c96b641468207c87fdfb2121c'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:30.362244 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-eba6621c-ff82-427c-81e1-f3080c6dd675 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a95dfe4c96b641468207c87fdfb2121c'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:30.375653 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1463/2899] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:38:30 2022] DELETE /identity/v3/users/a95dfe4c96b641468207c87fdfb2121c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:30.639699 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e991f8-fc67-4513-a3b0-44cde88cd823 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:30.641802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-88e991f8-fc67-4513-a3b0-44cde88cd823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:30.654831 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e991f8-fc67-4513-a3b0-44cde88cd823 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:30.712312 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f432e0-ba74-4f87-b218-d5fb2a448400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:30.713014 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f432e0-ba74-4f87-b218-d5fb2a448400 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:30.713186 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f432e0-ba74-4f87-b218-d5fb2a448400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:30.713451 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f432e0-ba74-4f87-b218-d5fb2a448400 admin admin] PATH_INFO: `/v3/projects/45a2f920cce2467fa7103fe8f5412155` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:30.714539 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f432e0-ba74-4f87-b218-d5fb2a448400 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45a2f920cce2467fa7103fe8f5412155)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:30.716462 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f432e0-ba74-4f87-b218-d5fb2a448400 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:30.761342 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b4f432e0-ba74-4f87-b218-d5fb2a448400 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45a2f920cce2467fa7103fe8f5412155', 'request_id': 'req-b4f432e0-ba74-4f87-b218-d5fb2a448400'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:30.768452 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b4f432e0-ba74-4f87-b218-d5fb2a448400 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45a2f920cce2467fa7103fe8f5412155', 'request_id': 'req-b4f432e0-ba74-4f87-b218-d5fb2a448400'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:30.773798 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b4f432e0-ba74-4f87-b218-d5fb2a448400 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45a2f920cce2467fa7103fe8f5412155', 'request_id': 'req-b4f432e0-ba74-4f87-b218-d5fb2a448400'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:30.784437 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b4f432e0-ba74-4f87-b218-d5fb2a448400 admin admin] The token cache is being invalidate because project 45a2f920cce2467fa7103fe8f5412155 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:38:30.784718 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b4f432e0-ba74-4f87-b218-d5fb2a448400 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:30.785327 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-b4f432e0-ba74-4f87-b218-d5fb2a448400 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:30.786773 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1437/2900] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:38:30 2022] DELETE /identity/v3/projects/45a2f920cce2467fa7103fe8f5412155 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:30.790731 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6621c-ff82-427c-81e1-f3080c6dd675 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:30.792129 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-eba6621c-ff82-427c-81e1-f3080c6dd675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:30.804933 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6621c-ff82-427c-81e1-f3080c6dd675 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:30.859054 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6e682-41dc-44ce-a6b0-3f9af2dc07be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:30.859704 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6e682-41dc-44ce-a6b0-3f9af2dc07be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:30.859984 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6e682-41dc-44ce-a6b0-3f9af2dc07be admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:30.860258 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6e682-41dc-44ce-a6b0-3f9af2dc07be admin admin] PATH_INFO: `/v3/users/95d86235494a422eaede673e34ed3f61` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:30.861347 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6e682-41dc-44ce-a6b0-3f9af2dc07be admin admin] RBAC: Authorizing `identity:delete_user(user_id=95d86235494a422eaede673e34ed3f61)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:30.863244 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6e682-41dc-44ce-a6b0-3f9af2dc07be admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:30.906239 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-dcc6e682-41dc-44ce-a6b0-3f9af2dc07be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95d86235494a422eaede673e34ed3f61'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:30.913132 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-dcc6e682-41dc-44ce-a6b0-3f9af2dc07be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95d86235494a422eaede673e34ed3f61'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:30.913776 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-dcc6e682-41dc-44ce-a6b0-3f9af2dc07be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95d86235494a422eaede673e34ed3f61'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:30.914376 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-dcc6e682-41dc-44ce-a6b0-3f9af2dc07be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95d86235494a422eaede673e34ed3f61'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:30.926729 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-dcc6e682-41dc-44ce-a6b0-3f9af2dc07be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95d86235494a422eaede673e34ed3f61'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:30.930298 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:30.942710 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1464/2901] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:38:30 2022] DELETE /identity/v3/users/95d86235494a422eaede673e34ed3f61 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:31.216513 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f432e0-ba74-4f87-b218-d5fb2a448400 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:31.218575 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-b4f432e0-ba74-4f87-b218-d5fb2a448400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:31.233686 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f432e0-ba74-4f87-b218-d5fb2a448400 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:31.242768 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:31.298204 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b144a0-0228-449a-9eb7-81a7fe666fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:31.298762 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b144a0-0228-449a-9eb7-81a7fe666fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:31.299042 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b144a0-0228-449a-9eb7-81a7fe666fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:31.299319 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b144a0-0228-449a-9eb7-81a7fe666fa4 admin admin] PATH_INFO: `/v3/projects/63736e0f8ba94e43988e4edb4b3f67df` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:31.300482 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b144a0-0228-449a-9eb7-81a7fe666fa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63736e0f8ba94e43988e4edb4b3f67df)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:31.302403 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b144a0-0228-449a-9eb7-81a7fe666fa4 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:31.346894 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-f6b144a0-0228-449a-9eb7-81a7fe666fa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63736e0f8ba94e43988e4edb4b3f67df', 'request_id': 'req-f6b144a0-0228-449a-9eb7-81a7fe666fa4'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:31.353549 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-f6b144a0-0228-449a-9eb7-81a7fe666fa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63736e0f8ba94e43988e4edb4b3f67df', 'request_id': 'req-f6b144a0-0228-449a-9eb7-81a7fe666fa4'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:31.359894 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-f6b144a0-0228-449a-9eb7-81a7fe666fa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63736e0f8ba94e43988e4edb4b3f67df', 'request_id': 'req-f6b144a0-0228-449a-9eb7-81a7fe666fa4'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:31.371325 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-f6b144a0-0228-449a-9eb7-81a7fe666fa4 admin admin] The token cache is being invalidate because project 63736e0f8ba94e43988e4edb4b3f67df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:38:31.371637 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-f6b144a0-0228-449a-9eb7-81a7fe666fa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:31.372280 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-f6b144a0-0228-449a-9eb7-81a7fe666fa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:31.373725 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1438/2902] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:38:31 2022] DELETE /identity/v3/projects/63736e0f8ba94e43988e4edb4b3f67df => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:31.378592 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338fbca1-a45b-4c1f-9d2c-f4e518e3683a None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:31.378881 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338fbca1-a45b-4c1f-9d2c-f4e518e3683a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:31.379158 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338fbca1-a45b-4c1f-9d2c-f4e518e3683a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:31.398810 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-338fbca1-a45b-4c1f-9d2c-f4e518e3683a None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:31.400055 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-338fbca1-a45b-4c1f-9d2c-f4e518e3683a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:31.458300 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1465/2903] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:38:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:31.466060 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48a166-2ce2-4056-926d-93bdd191dcf1 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:31.466337 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48a166-2ce2-4056-926d-93bdd191dcf1 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:31.466629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48a166-2ce2-4056-926d-93bdd191dcf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:31.486829 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-eb48a166-2ce2-4056-926d-93bdd191dcf1 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:31.488041 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-eb48a166-2ce2-4056-926d-93bdd191dcf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:31.495760 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1439/2904] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:38:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:31.499913 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338fbca1-a45b-4c1f-9d2c-f4e518e3683a None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:31.509606 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338fbca1-a45b-4c1f-9d2c-f4e518e3683a None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:31.514686 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd63f4d-3ce7-4560-961e-10aa19434e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:31.515279 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd63f4d-3ce7-4560-961e-10aa19434e79 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:31.515586 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd63f4d-3ce7-4560-961e-10aa19434e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:31.515867 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd63f4d-3ce7-4560-961e-10aa19434e79 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:31.516775 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd63f4d-3ce7-4560-961e-10aa19434e79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:38:31.517079 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd63f4d-3ce7-4560-961e-10aa19434e79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:31.518672 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd63f4d-3ce7-4560-961e-10aa19434e79 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:31.530673 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1466/2905] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:38:31.534785 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48a166-2ce2-4056-926d-93bdd191dcf1 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:31.544010 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48a166-2ce2-4056-926d-93bdd191dcf1 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:31.548774 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caab306-ebd8-4915-8ef2-865517e30870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:31.549291 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caab306-ebd8-4915-8ef2-865517e30870 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:31.549571 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caab306-ebd8-4915-8ef2-865517e30870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:31.549843 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caab306-ebd8-4915-8ef2-865517e30870 admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:31.550468 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caab306-ebd8-4915-8ef2-865517e30870 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:31.552388 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caab306-ebd8-4915-8ef2-865517e30870 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:31.579556 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1440/2906] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:38:31 2022] POST /identity/v3/projects => generated 391 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:38:31.584091 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd63f4d-3ce7-4560-961e-10aa19434e79 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:31.594456 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd63f4d-3ce7-4560-961e-10aa19434e79 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:31.649853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e217e-d6be-4fd4-8af9-38509691c3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:31.650456 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e217e-d6be-4fd4-8af9-38509691c3a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:31.650735 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e217e-d6be-4fd4-8af9-38509691c3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:31.651007 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e217e-d6be-4fd4-8af9-38509691c3a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:31.651676 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86e217e-d6be-4fd4-8af9-38509691c3a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:31.653588 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86e217e-d6be-4fd4-8af9-38509691c3a4 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:31.674436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1467/2907] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:38:31 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:38:31.678529 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caab306-ebd8-4915-8ef2-865517e30870 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:31.688810 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caab306-ebd8-4915-8ef2-865517e30870 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:31.693476 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b2f98-acd9-4bb1-978e-75de234d34fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:31.694068 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b2f98-acd9-4bb1-978e-75de234d34fb admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:31.694347 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b2f98-acd9-4bb1-978e-75de234d34fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:31.694620 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b2f98-acd9-4bb1-978e-75de234d34fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:31.695260 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139b2f98-acd9-4bb1-978e-75de234d34fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:31.696866 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139b2f98-acd9-4bb1-978e-75de234d34fb admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:31.707379 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1441/2908] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:38:31.711074 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e217e-d6be-4fd4-8af9-38509691c3a4 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:31.721543 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e217e-d6be-4fd4-8af9-38509691c3a4 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:31.726287 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79675a42-9d0c-4210-825a-c7bf711e1ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:31.726877 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79675a42-9d0c-4210-825a-c7bf711e1ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:31.727154 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79675a42-9d0c-4210-825a-c7bf711e1ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:31.727426 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79675a42-9d0c-4210-825a-c7bf711e1ef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:31.728089 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79675a42-9d0c-4210-825a-c7bf711e1ef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:31.729656 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79675a42-9d0c-4210-825a-c7bf711e1ef4 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:31.739378 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1468/2909] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:38:31.743267 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b2f98-acd9-4bb1-978e-75de234d34fb admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:31.753321 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b2f98-acd9-4bb1-978e-75de234d34fb admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:31.758358 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2daeb-7cfe-4749-a21c-68deebdf98d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:31.758996 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2daeb-7cfe-4749-a21c-68deebdf98d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:31.759276 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2daeb-7cfe-4749-a21c-68deebdf98d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:31.759579 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2daeb-7cfe-4749-a21c-68deebdf98d5 admin admin] PATH_INFO: `/v3/projects/7ae0fded5f6e420d85c701c91e24999a/users/fe0029fdd3ab4651895121b85f2f19fe/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:31.774634 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd2daeb-7cfe-4749-a21c-68deebdf98d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ae0fded5f6e420d85c701c91e24999a, user_id=fe0029fdd3ab4651895121b85f2f19fe, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:31.777308 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd2daeb-7cfe-4749-a21c-68deebdf98d5 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:31.790620 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1442/2910] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:38:31 2022] PUT /identity/v3/projects/7ae0fded5f6e420d85c701c91e24999a/users/fe0029fdd3ab4651895121b85f2f19fe/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:31.802528 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eeff34-8e05-4cd3-b519-46a6978bd196 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:31.803033 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eeff34-8e05-4cd3-b519-46a6978bd196 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:31.803033 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eeff34-8e05-4cd3-b519-46a6978bd196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:31.831176 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-15eeff34-8e05-4cd3-b519-46a6978bd196 None None] MFA Rules not processed for user `fe0029fdd3ab4651895121b85f2f19fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:31.832411 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-15eeff34-8e05-4cd3-b519-46a6978bd196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:31.880170 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:31.898183 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1469/2911] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:38:31 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:31.908731 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2daeb-7cfe-4749-a21c-68deebdf98d5 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:31.910405 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-5bd2daeb-7cfe-4749-a21c-68deebdf98d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:31.952810 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:31.969917 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2daeb-7cfe-4749-a21c-68deebdf98d5 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:31.973638 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0fc7e5-17b4-4d56-87b6-875e6c3170a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a47a0d9bfd473cbf0147d7801442d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:31.974236 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fc7e5-17b4-4d56-87b6-875e6c3170a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:31.974518 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fc7e5-17b4-4d56-87b6-875e6c3170a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:31.974793 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fc7e5-17b4-4d56-87b6-875e6c3170a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:31.983192 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0fc7e5-17b4-4d56-87b6-875e6c3170a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:31.984651 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0fc7e5-17b4-4d56-87b6-875e6c3170a8 service cinder] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:31.993417 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1443/2912] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:38:31 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:38:41.895261 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:41.989358 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=9 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:53.313903 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eeff34-8e05-4cd3-b519-46a6978bd196 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:53.381109 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eeff34-8e05-4cd3-b519-46a6978bd196 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:53.385820 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a57403-b2a1-45a0-9794-9799d9181eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:53.386483 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a57403-b2a1-45a0-9794-9799d9181eaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:53.386766 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a57403-b2a1-45a0-9794-9799d9181eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:53.387043 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a57403-b2a1-45a0-9794-9799d9181eaf admin admin] PATH_INFO: `/v3/users/fe0029fdd3ab4651895121b85f2f19fe` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:53.388286 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a57403-b2a1-45a0-9794-9799d9181eaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe0029fdd3ab4651895121b85f2f19fe)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:53.390267 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a57403-b2a1-45a0-9794-9799d9181eaf admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:53.434826 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-02a57403-b2a1-45a0-9794-9799d9181eaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe0029fdd3ab4651895121b85f2f19fe'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:53.442418 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-02a57403-b2a1-45a0-9794-9799d9181eaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe0029fdd3ab4651895121b85f2f19fe'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:53.443078 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-02a57403-b2a1-45a0-9794-9799d9181eaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe0029fdd3ab4651895121b85f2f19fe'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:53.443764 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-02a57403-b2a1-45a0-9794-9799d9181eaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe0029fdd3ab4651895121b85f2f19fe'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:53.455579 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-02a57403-b2a1-45a0-9794-9799d9181eaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe0029fdd3ab4651895121b85f2f19fe'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:53.468394 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1470/2913] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:38:53 2022] DELETE /identity/v3/users/fe0029fdd3ab4651895121b85f2f19fe => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:53.472916 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a043b6f-fe9d-4932-aadb-50edaede1802 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:53.473408 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a043b6f-fe9d-4932-aadb-50edaede1802 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:53.473520 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a043b6f-fe9d-4932-aadb-50edaede1802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:53.493235 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-1a043b6f-fe9d-4932-aadb-50edaede1802 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:53.494481 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-1a043b6f-fe9d-4932-aadb-50edaede1802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:53.553388 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1444/2914] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:53.563756 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a57403-b2a1-45a0-9794-9799d9181eaf admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:53.565129 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-02a57403-b2a1-45a0-9794-9799d9181eaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:53.620218 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a57403-b2a1-45a0-9794-9799d9181eaf admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:53.623958 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab138135-32e6-4636-b85f-87d902221f88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:53.625073 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab138135-32e6-4636-b85f-87d902221f88 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:53.625356 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab138135-32e6-4636-b85f-87d902221f88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:53.625655 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab138135-32e6-4636-b85f-87d902221f88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:53.634543 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab138135-32e6-4636-b85f-87d902221f88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:53.636037 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab138135-32e6-4636-b85f-87d902221f88 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:53.646128 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1471/2915] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:38:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:38:53.905523 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a043b6f-fe9d-4932-aadb-50edaede1802 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:53.907030 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-1a043b6f-fe9d-4932-aadb-50edaede1802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:53.920700 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a043b6f-fe9d-4932-aadb-50edaede1802 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:53.926092 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c155e71-e34c-4db9-a7a9-5e5dd5b7403e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:53.926773 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c155e71-e34c-4db9-a7a9-5e5dd5b7403e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:53.927190 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c155e71-e34c-4db9-a7a9-5e5dd5b7403e admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:53.927469 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c155e71-e34c-4db9-a7a9-5e5dd5b7403e admin admin] PATH_INFO: `/v3/projects/7ae0fded5f6e420d85c701c91e24999a` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:53.928737 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c155e71-e34c-4db9-a7a9-5e5dd5b7403e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ae0fded5f6e420d85c701c91e24999a)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:53.930581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c155e71-e34c-4db9-a7a9-5e5dd5b7403e admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:53.978455 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3c155e71-e34c-4db9-a7a9-5e5dd5b7403e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ae0fded5f6e420d85c701c91e24999a', 'request_id': 'req-3c155e71-e34c-4db9-a7a9-5e5dd5b7403e'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:53.986267 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3c155e71-e34c-4db9-a7a9-5e5dd5b7403e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ae0fded5f6e420d85c701c91e24999a', 'request_id': 'req-3c155e71-e34c-4db9-a7a9-5e5dd5b7403e'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:53.992194 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3c155e71-e34c-4db9-a7a9-5e5dd5b7403e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ae0fded5f6e420d85c701c91e24999a', 'request_id': 'req-3c155e71-e34c-4db9-a7a9-5e5dd5b7403e'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:54.003902 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3c155e71-e34c-4db9-a7a9-5e5dd5b7403e admin admin] The token cache is being invalidate because project 7ae0fded5f6e420d85c701c91e24999a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:38:54.004188 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3c155e71-e34c-4db9-a7a9-5e5dd5b7403e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:54.004819 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-3c155e71-e34c-4db9-a7a9-5e5dd5b7403e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:54.006284 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1445/2916] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:38:53 2022] DELETE /identity/v3/projects/7ae0fded5f6e420d85c701c91e24999a => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:54.011094 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb6b4e-944d-4c07-81dc-a7d3efd5371f None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:54.011380 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb6b4e-944d-4c07-81dc-a7d3efd5371f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:54.011790 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb6b4e-944d-4c07-81dc-a7d3efd5371f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:54.034970 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-67fb6b4e-944d-4c07-81dc-a7d3efd5371f None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:54.036185 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-67fb6b4e-944d-4c07-81dc-a7d3efd5371f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:54.053024 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:54.095507 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1472/2917] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:38:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:54.103241 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda6c20-9507-4df9-a0c2-69c42247b3e3 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:54.103586 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda6c20-9507-4df9-a0c2-69c42247b3e3 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:54.103862 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda6c20-9507-4df9-a0c2-69c42247b3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:54.124736 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-0dda6c20-9507-4df9-a0c2-69c42247b3e3 None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:54.125920 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-0dda6c20-9507-4df9-a0c2-69c42247b3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:54.133659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1446/2918] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:38:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:54.137705 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb6b4e-944d-4c07-81dc-a7d3efd5371f None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:54.147895 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb6b4e-944d-4c07-81dc-a7d3efd5371f None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:54.152398 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245639c0-36ad-4eef-9233-008d76b528ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:54.152985 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245639c0-36ad-4eef-9233-008d76b528ad admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:54.153264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245639c0-36ad-4eef-9233-008d76b528ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:54.153540 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245639c0-36ad-4eef-9233-008d76b528ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:54.154174 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245639c0-36ad-4eef-9233-008d76b528ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:38:54.154475 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245639c0-36ad-4eef-9233-008d76b528ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:54.156185 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245639c0-36ad-4eef-9233-008d76b528ad admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:54.167390 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1473/2919] 10.222.0.16 () {58 vars in 1153 bytes} [Mon Sep 5 12:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:38:54.171893 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda6c20-9507-4df9-a0c2-69c42247b3e3 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:54.180798 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda6c20-9507-4df9-a0c2-69c42247b3e3 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:54.185495 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f01cb-39fd-4678-9cb2-de33f00d9496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:54.186079 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f01cb-39fd-4678-9cb2-de33f00d9496 admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:54.186350 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f01cb-39fd-4678-9cb2-de33f00d9496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:54.186614 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f01cb-39fd-4678-9cb2-de33f00d9496 admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:54.187836 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54f01cb-39fd-4678-9cb2-de33f00d9496 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:54.189702 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54f01cb-39fd-4678-9cb2-de33f00d9496 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:54.215952 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1447/2920] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:38:54 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:38:54.219945 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245639c0-36ad-4eef-9233-008d76b528ad admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:54.229730 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245639c0-36ad-4eef-9233-008d76b528ad admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:54.286050 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91ac1f-a738-4cb1-8e39-87ea68358dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:54.286649 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91ac1f-a738-4cb1-8e39-87ea68358dfa admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:54.286925 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91ac1f-a738-4cb1-8e39-87ea68358dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:54.287191 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91ac1f-a738-4cb1-8e39-87ea68358dfa admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:54.287859 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd91ac1f-a738-4cb1-8e39-87ea68358dfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:54.289756 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd91ac1f-a738-4cb1-8e39-87ea68358dfa admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:54.312756 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1474/2921] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:38:54 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:38:54.316898 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f01cb-39fd-4678-9cb2-de33f00d9496 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:54.327081 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f01cb-39fd-4678-9cb2-de33f00d9496 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:54.331761 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b639bed-1f1e-494c-954e-dbe6b8933571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:54.332356 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b639bed-1f1e-494c-954e-dbe6b8933571 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:54.332635 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b639bed-1f1e-494c-954e-dbe6b8933571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:54.332907 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b639bed-1f1e-494c-954e-dbe6b8933571 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:54.333548 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b639bed-1f1e-494c-954e-dbe6b8933571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:54.335127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b639bed-1f1e-494c-954e-dbe6b8933571 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:54.345801 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1448/2922] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:38:54.349306 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91ac1f-a738-4cb1-8e39-87ea68358dfa admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:54.359625 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91ac1f-a738-4cb1-8e39-87ea68358dfa admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:54.364316 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93acb986-82de-46cf-b5c7-f91bf84c9cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:54.364898 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93acb986-82de-46cf-b5c7-f91bf84c9cdb admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:54.365181 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93acb986-82de-46cf-b5c7-f91bf84c9cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:54.365456 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93acb986-82de-46cf-b5c7-f91bf84c9cdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:54.366436 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93acb986-82de-46cf-b5c7-f91bf84c9cdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:54.368227 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93acb986-82de-46cf-b5c7-f91bf84c9cdb admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:54.378745 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1475/2923] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:38:54.382802 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b639bed-1f1e-494c-954e-dbe6b8933571 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:54.392972 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b639bed-1f1e-494c-954e-dbe6b8933571 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:54.397629 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4737e2-3956-4ba9-8d0a-6f7e89eb8f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:54.398262 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4737e2-3956-4ba9-8d0a-6f7e89eb8f41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:54.398542 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4737e2-3956-4ba9-8d0a-6f7e89eb8f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:54.398817 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4737e2-3956-4ba9-8d0a-6f7e89eb8f41 admin admin] PATH_INFO: `/v3/projects/7282ae16ef3d42979dfe9616a6b16238/users/a2556517dad2454c8cbbf26c5a9ea764/roles/ede2986592354e9888dde3123b2d8c82` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:54.413513 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4737e2-3956-4ba9-8d0a-6f7e89eb8f41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7282ae16ef3d42979dfe9616a6b16238, user_id=a2556517dad2454c8cbbf26c5a9ea764, role_id=ede2986592354e9888dde3123b2d8c82)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:54.416189 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4737e2-3956-4ba9-8d0a-6f7e89eb8f41 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:54.428814 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1449/2924] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:38:54 2022] PUT /identity/v3/projects/7282ae16ef3d42979dfe9616a6b16238/users/a2556517dad2454c8cbbf26c5a9ea764/roles/ede2986592354e9888dde3123b2d8c82 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:38:54.433875 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afdc26-79ed-4447-8a2d-bcd016f25e4c None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:54.434182 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afdc26-79ed-4447-8a2d-bcd016f25e4c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:54.434455 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afdc26-79ed-4447-8a2d-bcd016f25e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:54.454121 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-f4afdc26-79ed-4447-8a2d-bcd016f25e4c None None] MFA Rules not processed for user `78575c7ceaa64d1b86e8eef0293f1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:54.455318 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f4afdc26-79ed-4447-8a2d-bcd016f25e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:54.513712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1476/2925] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:38:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:54.524901 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4737e2-3956-4ba9-8d0a-6f7e89eb8f41 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:54.526712 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-fd4737e2-3956-4ba9-8d0a-6f7e89eb8f41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:54.550577 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:54.588069 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4737e2-3956-4ba9-8d0a-6f7e89eb8f41 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:54.591728 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9af96f-278b-4246-8001-080479bfd935 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:54.592328 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9af96f-278b-4246-8001-080479bfd935 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:54.592613 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9af96f-278b-4246-8001-080479bfd935 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:54.592885 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9af96f-278b-4246-8001-080479bfd935 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:54.601755 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9af96f-278b-4246-8001-080479bfd935 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:54.603127 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9af96f-278b-4246-8001-080479bfd935 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:54.612468 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1450/2926] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:38:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:38:58.549026 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080866f-ac3d-4829-b40b-dc374921ff20 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:58.549537 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080866f-ac3d-4829-b40b-dc374921ff20 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:58.549537 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080866f-ac3d-4829-b40b-dc374921ff20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:58.579831 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-b080866f-ac3d-4829-b40b-dc374921ff20 None None] MFA Rules not processed for user `a2556517dad2454c8cbbf26c5a9ea764`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:58.581070 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b080866f-ac3d-4829-b40b-dc374921ff20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:58.600813 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:58.659689 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1477/2927] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:38:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:58.661574 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9af96f-278b-4246-8001-080479bfd935 service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:58.671556 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9af96f-278b-4246-8001-080479bfd935 service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:58.676684 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f541c0-5d68-4610-bc2f-0d8e3b3cf49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:58.677290 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f541c0-5d68-4610-bc2f-0d8e3b3cf49a admin admin] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:58.677572 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f541c0-5d68-4610-bc2f-0d8e3b3cf49a admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:58.677848 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f541c0-5d68-4610-bc2f-0d8e3b3cf49a admin admin] PATH_INFO: `/v3/projects` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:58.678488 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f541c0-5d68-4610-bc2f-0d8e3b3cf49a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:58.680522 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f541c0-5d68-4610-bc2f-0d8e3b3cf49a admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:58.708969 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1451/2928] 10.222.0.16 () {60 vars in 1153 bytes} [Mon Sep 5 12:38:58 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:38:58.712959 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080866f-ac3d-4829-b40b-dc374921ff20 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:58.724665 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080866f-ac3d-4829-b40b-dc374921ff20 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:58.789789 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d9976-fd24-4959-814d-c842dea3527f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:58.790495 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d9976-fd24-4959-814d-c842dea3527f admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:58.790860 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d9976-fd24-4959-814d-c842dea3527f admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:58.791197 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d9976-fd24-4959-814d-c842dea3527f admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:58.791887 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105d9976-fd24-4959-814d-c842dea3527f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:58.793798 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105d9976-fd24-4959-814d-c842dea3527f admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:58.816214 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1478/2929] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Sep 5 12:38:58 2022] POST /identity/v3/users => generated 298 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:38:58.820286 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f541c0-5d68-4610-bc2f-0d8e3b3cf49a admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:58.831926 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f541c0-5d68-4610-bc2f-0d8e3b3cf49a admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:58.836664 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ef19f-2093-41ba-9d81-6c54cf6d43fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:58.837277 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ef19f-2093-41ba-9d81-6c54cf6d43fe admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:58.837558 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ef19f-2093-41ba-9d81-6c54cf6d43fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:58.837828 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ef19f-2093-41ba-9d81-6c54cf6d43fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:58.838477 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ef19f-2093-41ba-9d81-6c54cf6d43fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:58.840106 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ef19f-2093-41ba-9d81-6c54cf6d43fe admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:58.851441 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1452/2930] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:38:58.855618 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d9976-fd24-4959-814d-c842dea3527f admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:58.865756 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d9976-fd24-4959-814d-c842dea3527f admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:58.871515 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f424f0-98f6-43e7-bf11-80beb1703f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:58.872264 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f424f0-98f6-43e7-bf11-80beb1703f40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:58.872541 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f424f0-98f6-43e7-bf11-80beb1703f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:58.872813 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f424f0-98f6-43e7-bf11-80beb1703f40 admin admin] PATH_INFO: `/v3/projects/aa75da69764f4c56b2ede4a2057b1fdc/users/a3b0bfcc72d34e53ae4f95abba3cd45c/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:58.892410 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f424f0-98f6-43e7-bf11-80beb1703f40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa75da69764f4c56b2ede4a2057b1fdc, user_id=a3b0bfcc72d34e53ae4f95abba3cd45c, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:58.895157 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f424f0-98f6-43e7-bf11-80beb1703f40 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:58.909857 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1479/2931] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:38:58 2022] PUT /identity/v3/projects/aa75da69764f4c56b2ede4a2057b1fdc/users/a3b0bfcc72d34e53ae4f95abba3cd45c/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:58.913831 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ef19f-2093-41ba-9d81-6c54cf6d43fe admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:58.924772 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ef19f-2093-41ba-9d81-6c54cf6d43fe admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:58.987446 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761d35e-b7a6-4e28-87c8-9dbc2a2654f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:58.988091 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761d35e-b7a6-4e28-87c8-9dbc2a2654f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:58.988413 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761d35e-b7a6-4e28-87c8-9dbc2a2654f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:58.988697 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761d35e-b7a6-4e28-87c8-9dbc2a2654f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:58.989344 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3761d35e-b7a6-4e28-87c8-9dbc2a2654f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:58.990929 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3761d35e-b7a6-4e28-87c8-9dbc2a2654f7 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:59.004582 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1453/2932] 10.222.0.16 () {58 vars in 1122 bytes} [Mon Sep 5 12:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:38:59.005805 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f424f0-98f6-43e7-bf11-80beb1703f40 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:59.017571 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f424f0-98f6-43e7-bf11-80beb1703f40 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:59.022406 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637dfcb-0bbc-4b1f-8649-c5b4480d9476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:59.023070 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637dfcb-0bbc-4b1f-8649-c5b4480d9476 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:59.023353 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637dfcb-0bbc-4b1f-8649-c5b4480d9476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:59.023666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637dfcb-0bbc-4b1f-8649-c5b4480d9476 admin admin] PATH_INFO: `/v3/projects/aa75da69764f4c56b2ede4a2057b1fdc/users/a3b0bfcc72d34e53ae4f95abba3cd45c/roles/618c48706f7f445abeddf7eb2f8e927a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:59.025649 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2637dfcb-0bbc-4b1f-8649-c5b4480d9476 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa75da69764f4c56b2ede4a2057b1fdc, user_id=a3b0bfcc72d34e53ae4f95abba3cd45c, role_id=618c48706f7f445abeddf7eb2f8e927a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:59.028330 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2637dfcb-0bbc-4b1f-8649-c5b4480d9476 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:59.039243 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1480/2933] 10.222.0.16 () {60 vars in 1483 bytes} [Mon Sep 5 12:38:59 2022] PUT /identity/v3/projects/aa75da69764f4c56b2ede4a2057b1fdc/users/a3b0bfcc72d34e53ae4f95abba3cd45c/roles/618c48706f7f445abeddf7eb2f8e927a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:39:03.559155 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1dcf68-1294-4051-bb0a-3cb75d56a8f0 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:39:03.559648 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1dcf68-1294-4051-bb0a-3cb75d56a8f0 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:39:03.559812 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1dcf68-1294-4051-bb0a-3cb75d56a8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:39:03.598758 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-fb1dcf68-1294-4051-bb0a-3cb75d56a8f0 None None] MFA Rules not processed for user `a3b0bfcc72d34e53ae4f95abba3cd45c`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:39:03.600564 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-fb1dcf68-1294-4051-bb0a-3cb75d56a8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:39:03.682131 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1454/2934] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:39:03 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:39:03.700277 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637dfcb-0bbc-4b1f-8649-c5b4480d9476 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:39:03.702300 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2637dfcb-0bbc-4b1f-8649-c5b4480d9476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:39:03.792555 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637dfcb-0bbc-4b1f-8649-c5b4480d9476 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:39:03.796644 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9af3f3-09b0-4226-9f41-0619b0264314 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed75c0be809d4a0eb0e2ccc59805b1ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:39:03.797322 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9af3f3-09b0-4226-9f41-0619b0264314 service glance] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:39:03.797670 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9af3f3-09b0-4226-9f41-0619b0264314 service glance] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:39:03.798019 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9af3f3-09b0-4226-9f41-0619b0264314 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:39:03.802561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:39:03.870327 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9af3f3-09b0-4226-9f41-0619b0264314 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:39:03.871981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9af3f3-09b0-4226-9f41-0619b0264314 service glance] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:39:03.883070 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1481/2935] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:39:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:39:04.035817 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfccf6-ac95-4c05-b046-974c6b9aa0b0 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:39:04.036107 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfccf6-ac95-4c05-b046-974c6b9aa0b0 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:39:04.036393 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfccf6-ac95-4c05-b046-974c6b9aa0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:39:04.054098 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:39:04.069176 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-66dfccf6-ac95-4c05-b046-974c6b9aa0b0 None None] MFA Rules not processed for user `8c5ae98336c441118d01f46a50f732d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:39:04.070503 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-66dfccf6-ac95-4c05-b046-974c6b9aa0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:39:04.137404 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1455/2936] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Sep 5 12:39:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:39:04.196677 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9af3f3-09b0-4226-9f41-0619b0264314 service glance] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:39:04.198111 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-0f9af3f3-09b0-4226-9f41-0619b0264314 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:39:04.270296 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9af3f3-09b0-4226-9f41-0619b0264314 service glance] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:39:04.275375 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0772ccfd-09ff-49a2-a301-c2a1ccce0643 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '857eb7657e7d45938baca17f91a16901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:39:04.276019 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772ccfd-09ff-49a2-a301-c2a1ccce0643 service swift] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:39:04.276334 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772ccfd-09ff-49a2-a301-c2a1ccce0643 service swift] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:39:04.276610 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772ccfd-09ff-49a2-a301-c2a1ccce0643 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:39:04.284677 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0772ccfd-09ff-49a2-a301-c2a1ccce0643 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:39:04.286248 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0772ccfd-09ff-49a2-a301-c2a1ccce0643 service swift] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:39:04.294191 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1482/2937] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Sep 5 12:39:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 05 12:39:13.627919 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f7e74-ac9b-4a59-82fe-7bfafdcd8ddc None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:39:13.628399 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f7e74-ac9b-4a59-82fe-7bfafdcd8ddc None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:39:13.628517 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f7e74-ac9b-4a59-82fe-7bfafdcd8ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:39:13.647242 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-911f7e74-ac9b-4a59-82fe-7bfafdcd8ddc None None] MFA Rules not processed for user `a2556517dad2454c8cbbf26c5a9ea764`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:39:13.648467 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-911f7e74-ac9b-4a59-82fe-7bfafdcd8ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:39:13.655072 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1456/2938] 10.222.0.16 () {58 vars in 958 bytes} [Mon Sep 5 12:39:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:39:13.910824 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0772ccfd-09ff-49a2-a301-c2a1ccce0643 service swift] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:39:13.964032 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0772ccfd-09ff-49a2-a301-c2a1ccce0643 service swift] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:39:13.967551 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7bb85f-f535-4089-b1bf-764291e16386 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:39:13.968655 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7bb85f-f535-4089-b1bf-764291e16386 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:39:13.968935 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7bb85f-f535-4089-b1bf-764291e16386 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:39:13.969209 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7bb85f-f535-4089-b1bf-764291e16386 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:39:13.970929 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ee7bb85f-f535-4089-b1bf-764291e16386 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:39:13.980241 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7bb85f-f535-4089-b1bf-764291e16386 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:39:13.981657 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7bb85f-f535-4089-b1bf-764291e16386 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:39:13.990173 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1483/2939] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:39:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:39:15.322783 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f7e74-ac9b-4a59-82fe-7bfafdcd8ddc None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:39:15.324249 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-911f7e74-ac9b-4a59-82fe-7bfafdcd8ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:39:15.336179 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f7e74-ac9b-4a59-82fe-7bfafdcd8ddc None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:39:15.339713 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4a0c6-7bfa-435b-8a35-99434a4d289a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:39:15.340302 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4a0c6-7bfa-435b-8a35-99434a4d289a service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:39:15.340581 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4a0c6-7bfa-435b-8a35-99434a4d289a service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:39:15.340851 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4a0c6-7bfa-435b-8a35-99434a4d289a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:39:15.342518 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-2cd4a0c6-7bfa-435b-8a35-99434a4d289a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:39:15.445214 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd4a0c6-7bfa-435b-8a35-99434a4d289a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:39:15.446659 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd4a0c6-7bfa-435b-8a35-99434a4d289a service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:39:15.467064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1457/2940] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:39:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:39:16.582097 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7bb85f-f535-4089-b1bf-764291e16386 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:39:16.592281 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7bb85f-f535-4089-b1bf-764291e16386 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:39:16.596411 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09920c25-0ec9-4371-a258-f0760bc2e01a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:39:16.597014 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09920c25-0ec9-4371-a258-f0760bc2e01a service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:39:16.597295 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09920c25-0ec9-4371-a258-f0760bc2e01a service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:39:16.597853 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09920c25-0ec9-4371-a258-f0760bc2e01a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:39:16.599763 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-09920c25-0ec9-4371-a258-f0760bc2e01a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:39:16.614339 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09920c25-0ec9-4371-a258-f0760bc2e01a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:39:16.615806 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09920c25-0ec9-4371-a258-f0760bc2e01a service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:39:16.628530 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1484/2941] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:39:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:39:16.640400 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f2bb0-2986-40a9-a69c-52860a142374 None None] REQUEST_METHOD: `POST` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:39:16.640909 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f2bb0-2986-40a9-a69c-52860a142374 None None] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:39:16.641296 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f2bb0-2986-40a9-a69c-52860a142374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:39:16.660747 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.auth.core [None req-d13f2bb0-2986-40a9-a69c-52860a142374 None None] MFA Rules not processed for user `8c5ae98336c441118d01f46a50f732d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:39:16.661969 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-d13f2bb0-2986-40a9-a69c-52860a142374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:39:16.673234 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1458/2942] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Sep 5 12:39:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:39:16.691456 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09920c25-0ec9-4371-a258-f0760bc2e01a service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:39:16.703556 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09920c25-0ec9-4371-a258-f0760bc2e01a service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:39:16.708756 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d707a729-edfa-4aaa-9a81-4c2fc6499664 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '857eb7657e7d45938baca17f91a16901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:39:16.709369 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707a729-edfa-4aaa-9a81-4c2fc6499664 service swift] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:39:16.709630 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707a729-edfa-4aaa-9a81-4c2fc6499664 service swift] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:39:16.709900 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707a729-edfa-4aaa-9a81-4c2fc6499664 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:39:16.718607 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d707a729-edfa-4aaa-9a81-4c2fc6499664 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:39:16.720361 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d707a729-edfa-4aaa-9a81-4c2fc6499664 service swift] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:39:16.728710 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1485/2943] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Sep 5 12:39:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 12:39:26.655875 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=76 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:39:26.725268 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=95 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:39:28.279051 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f2bb0-2986-40a9-a69c-52860a142374 None None] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:39:28.290955 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f2bb0-2986-40a9-a69c-52860a142374 None None] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:39:28.294720 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bbdda7-d7d7-4412-8d7c-a2a3ac986614 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:39:28.295397 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbdda7-d7d7-4412-8d7c-a2a3ac986614 service neutron] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:39:28.296408 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbdda7-d7d7-4412-8d7c-a2a3ac986614 service neutron] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:39:28.296744 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbdda7-d7d7-4412-8d7c-a2a3ac986614 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:39:28.300692 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-a0bbdda7-d7d7-4412-8d7c-a2a3ac986614 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:39:28.311064 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bbdda7-d7d7-4412-8d7c-a2a3ac986614 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:39:28.313178 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bbdda7-d7d7-4412-8d7c-a2a3ac986614 service neutron] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:39:28.322083 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1459/2944] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:39:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:39:38.318596 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=6 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:39:45.754496 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d707a729-edfa-4aaa-9a81-4c2fc6499664 service swift] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:39:45.763803 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d707a729-edfa-4aaa-9a81-4c2fc6499664 service swift] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:39:45.767585 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31864b2c-a85e-482a-a80b-2a1832f6df12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f79599ed0b74bdbab198627ef72252a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:39:45.768168 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31864b2c-a85e-482a-a80b-2a1832f6df12 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:39:45.768442 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31864b2c-a85e-482a-a80b-2a1832f6df12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:39:45.768711 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31864b2c-a85e-482a-a80b-2a1832f6df12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:39:45.777277 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31864b2c-a85e-482a-a80b-2a1832f6df12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:39:45.778756 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31864b2c-a85e-482a-a80b-2a1832f6df12 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:39:45.788373 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1486/2945] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:39:45 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:39:55.783783 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=6 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:42:06.267853 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bbdda7-d7d7-4412-8d7c-a2a3ac986614 service neutron] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:06.274140 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-a0bbdda7-d7d7-4412-8d7c-a2a3ac986614 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:06.293261 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bbdda7-d7d7-4412-8d7c-a2a3ac986614 service neutron] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:06.299566 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91d760-d784-4161-b9d9-5b3a49177b7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:06.300612 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91d760-d784-4161-b9d9-5b3a49177b7f service nova] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:06.300761 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91d760-d784-4161-b9d9-5b3a49177b7f service nova] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:06.301102 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91d760-d784-4161-b9d9-5b3a49177b7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:06.303454 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-2c91d760-d784-4161-b9d9-5b3a49177b7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:06.317830 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91d760-d784-4161-b9d9-5b3a49177b7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:06.319511 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91d760-d784-4161-b9d9-5b3a49177b7f service nova] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:06.333613 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1460/2946] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:42:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:42:06.479118 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31864b2c-a85e-482a-a80b-2a1832f6df12 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:06.480820 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-31864b2c-a85e-482a-a80b-2a1832f6df12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:06.588933 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31864b2c-a85e-482a-a80b-2a1832f6df12 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:06.595805 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44432d05-566f-40a3-a40f-09f9bb230fb3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524bcfffcdfa4790885e0d4cbd66fa80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:06.596433 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44432d05-566f-40a3-a40f-09f9bb230fb3 service placement] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:06.596712 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44432d05-566f-40a3-a40f-09f9bb230fb3 service placement] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:06.596981 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44432d05-566f-40a3-a40f-09f9bb230fb3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:06.599973 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-44432d05-566f-40a3-a40f-09f9bb230fb3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:06.614594 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44432d05-566f-40a3-a40f-09f9bb230fb3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:06.616614 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44432d05-566f-40a3-a40f-09f9bb230fb3 service placement] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:06.628394 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1487/2947] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:42:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:42:12.802281 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91d760-d784-4161-b9d9-5b3a49177b7f service nova] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:12.813809 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91d760-d784-4161-b9d9-5b3a49177b7f service nova] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:12.819503 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f762ac5-329c-4244-b637-7f72ee218876 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda290418504f0db85d665479a05353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:12.820130 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f762ac5-329c-4244-b637-7f72ee218876 service nova] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:12.820407 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f762ac5-329c-4244-b637-7f72ee218876 service nova] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:12.820676 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f762ac5-329c-4244-b637-7f72ee218876 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:12.822435 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-9f762ac5-329c-4244-b637-7f72ee218876 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:12.836513 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f762ac5-329c-4244-b637-7f72ee218876 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:12.838084 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f762ac5-329c-4244-b637-7f72ee218876 service nova] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:12.853441 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1461/2948] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:42:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:42:13.573121 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c502a0-497c-467d-ae6b-f4d1c52ad02d None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:13.573600 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c502a0-497c-467d-ae6b-f4d1c52ad02d None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:13.573724 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c502a0-497c-467d-ae6b-f4d1c52ad02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:13.622346 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-60c502a0-497c-467d-ae6b-f4d1c52ad02d None None] MFA Rules not processed for user `8c5ae98336c441118d01f46a50f732d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:42:13.624204 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-60c502a0-497c-467d-ae6b-f4d1c52ad02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:13.640366 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1488/2949] 10.222.0.16 () {58 vars in 967 bytes} [Mon Sep 5 12:42:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:42:13.680148 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f762ac5-329c-4244-b637-7f72ee218876 service nova] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:13.701230 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f762ac5-329c-4244-b637-7f72ee218876 service nova] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:13.706285 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c992350-59f2-48b0-8ff2-3a6d51e86a2c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '857eb7657e7d45938baca17f91a16901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:13.706962 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c992350-59f2-48b0-8ff2-3a6d51e86a2c service swift] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:13.707242 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c992350-59f2-48b0-8ff2-3a6d51e86a2c service swift] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:13.707516 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c992350-59f2-48b0-8ff2-3a6d51e86a2c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:13.715370 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c992350-59f2-48b0-8ff2-3a6d51e86a2c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:13.717461 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c992350-59f2-48b0-8ff2-3a6d51e86a2c service swift] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:13.725334 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1462/2950] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Sep 5 12:42:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 12:42:17.049639 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c502a0-497c-467d-ae6b-f4d1c52ad02d None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:17.063020 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c502a0-497c-467d-ae6b-f4d1c52ad02d None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:17.073943 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05027223-a3aa-44ce-87a7-58afda900136 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524bcfffcdfa4790885e0d4cbd66fa80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:17.074920 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05027223-a3aa-44ce-87a7-58afda900136 service placement] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:17.075408 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05027223-a3aa-44ce-87a7-58afda900136 service placement] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:17.075914 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05027223-a3aa-44ce-87a7-58afda900136 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:17.078564 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-05027223-a3aa-44ce-87a7-58afda900136 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:17.096706 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05027223-a3aa-44ce-87a7-58afda900136 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:17.098332 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05027223-a3aa-44ce-87a7-58afda900136 service placement] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:17.110821 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1489/2951] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:42:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:42:20.244217 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c992350-59f2-48b0-8ff2-3a6d51e86a2c service swift] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:20.313680 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c992350-59f2-48b0-8ff2-3a6d51e86a2c service swift] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:20.318233 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8dd7c-688c-4be9-a17f-652fb79c0714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:20.318854 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8dd7c-688c-4be9-a17f-652fb79c0714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:20.319136 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8dd7c-688c-4be9-a17f-652fb79c0714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:20.319414 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8dd7c-688c-4be9-a17f-652fb79c0714 admin admin] PATH_INFO: `/v3/users/a2556517dad2454c8cbbf26c5a9ea764` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:20.320567 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8dd7c-688c-4be9-a17f-652fb79c0714 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2556517dad2454c8cbbf26c5a9ea764)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:20.322477 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8dd7c-688c-4be9-a17f-652fb79c0714 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:20.366806 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-4aa8dd7c-688c-4be9-a17f-652fb79c0714 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2556517dad2454c8cbbf26c5a9ea764'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:20.375647 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-4aa8dd7c-688c-4be9-a17f-652fb79c0714 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2556517dad2454c8cbbf26c5a9ea764'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:20.376254 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-4aa8dd7c-688c-4be9-a17f-652fb79c0714 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2556517dad2454c8cbbf26c5a9ea764'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:20.388423 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-4aa8dd7c-688c-4be9-a17f-652fb79c0714 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2556517dad2454c8cbbf26c5a9ea764'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:20.395288 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-4aa8dd7c-688c-4be9-a17f-652fb79c0714 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2556517dad2454c8cbbf26c5a9ea764'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:20.401023 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1463/2952] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:42:20 2022] DELETE /identity/v3/users/a2556517dad2454c8cbbf26c5a9ea764 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:42:20.566309 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05027223-a3aa-44ce-87a7-58afda900136 service placement] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:20.577536 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05027223-a3aa-44ce-87a7-58afda900136 service placement] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:20.633666 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e1cf1-39f3-4e2b-8515-d3d1b74edde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:20.634296 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e1cf1-39f3-4e2b-8515-d3d1b74edde3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:20.634577 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e1cf1-39f3-4e2b-8515-d3d1b74edde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:20.634851 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e1cf1-39f3-4e2b-8515-d3d1b74edde3 admin admin] PATH_INFO: `/v3/projects/7282ae16ef3d42979dfe9616a6b16238` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:20.636021 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e1cf1-39f3-4e2b-8515-d3d1b74edde3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7282ae16ef3d42979dfe9616a6b16238)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:20.637922 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e1cf1-39f3-4e2b-8515-d3d1b74edde3 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:20.677665 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:42:20.686279 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6f2e1cf1-39f3-4e2b-8515-d3d1b74edde3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7282ae16ef3d42979dfe9616a6b16238', 'request_id': 'req-6f2e1cf1-39f3-4e2b-8515-d3d1b74edde3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:20.691682 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6f2e1cf1-39f3-4e2b-8515-d3d1b74edde3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7282ae16ef3d42979dfe9616a6b16238', 'request_id': 'req-6f2e1cf1-39f3-4e2b-8515-d3d1b74edde3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:20.697560 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6f2e1cf1-39f3-4e2b-8515-d3d1b74edde3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7282ae16ef3d42979dfe9616a6b16238', 'request_id': 'req-6f2e1cf1-39f3-4e2b-8515-d3d1b74edde3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:20.709478 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6f2e1cf1-39f3-4e2b-8515-d3d1b74edde3 admin admin] The token cache is being invalidate because project 7282ae16ef3d42979dfe9616a6b16238 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:42:20.709762 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6f2e1cf1-39f3-4e2b-8515-d3d1b74edde3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:20.710373 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6f2e1cf1-39f3-4e2b-8515-d3d1b74edde3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:20.711858 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1490/2953] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:42:20 2022] DELETE /identity/v3/projects/7282ae16ef3d42979dfe9616a6b16238 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:42:20.716026 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8dd7c-688c-4be9-a17f-652fb79c0714 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:20.717970 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-4aa8dd7c-688c-4be9-a17f-652fb79c0714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:20.730615 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8dd7c-688c-4be9-a17f-652fb79c0714 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:20.759279 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:42:20.789374 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad83f3-1e8d-4b28-a6fb-f07a4f2e03a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:20.789997 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad83f3-1e8d-4b28-a6fb-f07a4f2e03a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:20.790279 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad83f3-1e8d-4b28-a6fb-f07a4f2e03a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:20.790553 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad83f3-1e8d-4b28-a6fb-f07a4f2e03a6 admin admin] PATH_INFO: `/v3/users/a3b0bfcc72d34e53ae4f95abba3cd45c` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:20.791717 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ad83f3-1e8d-4b28-a6fb-f07a4f2e03a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3b0bfcc72d34e53ae4f95abba3cd45c)` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:20.793624 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ad83f3-1e8d-4b28-a6fb-f07a4f2e03a6 admin admin] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:20.838399 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-d9ad83f3-1e8d-4b28-a6fb-f07a4f2e03a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3b0bfcc72d34e53ae4f95abba3cd45c'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:20.846204 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-d9ad83f3-1e8d-4b28-a6fb-f07a4f2e03a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3b0bfcc72d34e53ae4f95abba3cd45c'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:20.846807 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-d9ad83f3-1e8d-4b28-a6fb-f07a4f2e03a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3b0bfcc72d34e53ae4f95abba3cd45c'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:20.858831 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-d9ad83f3-1e8d-4b28-a6fb-f07a4f2e03a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3b0bfcc72d34e53ae4f95abba3cd45c'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:20.865524 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.notifications [None req-d9ad83f3-1e8d-4b28-a6fb-f07a4f2e03a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3b0bfcc72d34e53ae4f95abba3cd45c'} {{(pid=81541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:20.871254 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1464/2954] 10.222.0.16 () {58 vars in 1224 bytes} [Mon Sep 5 12:42:20 2022] DELETE /identity/v3/users/a3b0bfcc72d34e53ae4f95abba3cd45c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:42:21.042725 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e1cf1-39f3-4e2b-8515-d3d1b74edde3 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:21.044297 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6f2e1cf1-39f3-4e2b-8515-d3d1b74edde3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:21.059146 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e1cf1-39f3-4e2b-8515-d3d1b74edde3 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:21.117084 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc3836b-6714-4c20-b04c-f01c45018a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78575c7ceaa64d1b86e8eef0293f1e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '898d4110c367417eb23c75ef5e13bda3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:21.117634 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3836b-6714-4c20-b04c-f01c45018a95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:21.117908 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3836b-6714-4c20-b04c-f01c45018a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:21.118175 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3836b-6714-4c20-b04c-f01c45018a95 admin admin] PATH_INFO: `/v3/projects/aa75da69764f4c56b2ede4a2057b1fdc` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:21.119269 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc3836b-6714-4c20-b04c-f01c45018a95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa75da69764f4c56b2ede4a2057b1fdc)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:21.121185 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc3836b-6714-4c20-b04c-f01c45018a95 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:21.166027 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4fc3836b-6714-4c20-b04c-f01c45018a95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa75da69764f4c56b2ede4a2057b1fdc', 'request_id': 'req-4fc3836b-6714-4c20-b04c-f01c45018a95'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:21.171509 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4fc3836b-6714-4c20-b04c-f01c45018a95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa75da69764f4c56b2ede4a2057b1fdc', 'request_id': 'req-4fc3836b-6714-4c20-b04c-f01c45018a95'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:21.177505 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4fc3836b-6714-4c20-b04c-f01c45018a95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa75da69764f4c56b2ede4a2057b1fdc', 'request_id': 'req-4fc3836b-6714-4c20-b04c-f01c45018a95'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:21.189272 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4fc3836b-6714-4c20-b04c-f01c45018a95 admin admin] The token cache is being invalidate because project aa75da69764f4c56b2ede4a2057b1fdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:42:21.189561 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4fc3836b-6714-4c20-b04c-f01c45018a95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:21.190206 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4fc3836b-6714-4c20-b04c-f01c45018a95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:21.191740 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1491/2955] 10.222.0.16 () {58 vars in 1233 bytes} [Mon Sep 5 12:42:21 2022] DELETE /identity/v3/projects/aa75da69764f4c56b2ede4a2057b1fdc => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:42:30.867908 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=39,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:42:31.186324 dv-cinder-855189-1-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=61,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:47:11.407382 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad83f3-1e8d-4b28-a6fb-f07a4f2e03a6 admin admin] Authenticating user token {{(pid=81541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:47:11.408816 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-d9ad83f3-1e8d-4b28-a6fb-f07a4f2e03a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:47:11.518757 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad83f3-1e8d-4b28-a6fb-f07a4f2e03a6 admin admin] Validating token access rules against request {{(pid=81541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:47:11.577018 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a39788-ac44-4ffa-b2f7-75b62f6e4cc1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524bcfffcdfa4790885e0d4cbd66fa80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3178684d184ec880f17640d19d55b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:47:11.577618 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a39788-ac44-4ffa-b2f7-75b62f6e4cc1 service placement] REQUEST_METHOD: `GET` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:47:11.577900 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a39788-ac44-4ffa-b2f7-75b62f6e4cc1 service placement] SCRIPT_NAME: `/identity` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:47:11.578175 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a39788-ac44-4ffa-b2f7-75b62f6e4cc1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:47:11.579936 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.fernet_utils [None req-e4a39788-ac44-4ffa-b2f7-75b62f6e4cc1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:47:11.590977 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a39788-ac44-4ffa-b2f7-75b62f6e4cc1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:47:11.592564 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a39788-ac44-4ffa-b2f7-75b62f6e4cc1 service placement] RBAC: Authorization granted {{(pid=81541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:47:11.602378 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: [pid: 81541|app: 0|req: 1465/2956] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Sep 5 12:47:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:47:21.598113 dv-cinder-855189-1-iscsi devstack@keystone.service[81541]: DEBUG dbcounter [-] [81541] Writing DB stats keystone:SELECT=76 {{(pid=81541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}